A machine learning method and device for original text annotation based on OFD

By converting OFD files into multi-dimensional data element sets and generating xml data packets, efficient labeling and training of OFD files is achieved, solving the problems of low labeling efficiency and poor operation experience in the existing technology, and improving the labeling accuracy and experience, extending the range of data element sets.

CN118734810BActive Publication Date: 2024-11-15JIANGSU ZHONGWEI TECH SOFTWARE SYST
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411215683.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-15
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

The prior art is inefficient in labeling OFD file content and poor user operation experience, so it is impossible to label the OFD file directly, which affects the large model's tagging learning efficiency of text and picture information.

Method used

Provides an original text annotation machine learning method based on OFD. By uploading and converting files into OFD format, setting up multi-dimensional data element sets, loading and rendering files, recording tag information, generating data packets in xml format, and integrating them into OFD files. Support batch import of training models and displaying the annotation content in the reader to realize the jump and reverse traceability of the annotation.

Benefits of technology

It greatly reduces the workload of user labeling, improves the operation experience, expands the range of data element sets, makes labeling transcend semantics, supports multi-dimensional labeling such as fonts, font sizes, colors, regularities, and structures, and provides it when AI calculation results are reverse traced. Intuitive explanation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118734810B_ABST
    Figure CN118734810B_ABST
Patent Text Reader

Abstract

The present invention proposes an OFD-based original text annotation machine learning method and device, which uses the OFD format as a unified format for knowledge base files, fully utilizes the functional advantages of OFD files, gives full play to the role of the xml file package of OFD files, and innovatively injects training results into the xml file package of each OFD file, which can not only accurately bind the annotations to each file, but also use the structural rules of the OFD file xml to batch train other OFD files, greatly reducing the workload of user annotation, and using the OFD format as a unified format for knowledge base files, greatly expanding the scope of data element sets, so that annotations go beyond semantics and expand the scope of fonts, font sizes, colors, regular expressions, structures, positions, formats, etc.; the present invention allows users to directly annotate on OFD files, thereby improving the operating experience; and allows users to directly see the OFD original text when the AI ​​calculation results need to be traced back, thereby making the result interpretation more intuitive and reliable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an OFD-based original text annotation machine learning method and device. Background Art

[0002] On October 14, 2016, World Standards Day, the National Standardization Administration of China officially approved and issued the national standard GB / T 33190-2016 "Electronic File Storage and Exchange Format Layout Document" (OFD for short) based on independent technology. OFD, the abbreviation of OpenFixed-layout Document, is my country's independent and controllable electronic file layout document format, and the corresponding international standard is PDF. OFD layout files have fixed layouts, no layout changes, and what you see is what you get. They can be regarded as the "digital paper" of the computer age; they are the ideal document format for electronic document publishing, digital information dissemination and archiving.

[0003] Manual annotation is mostly used to annotate the content of OFD files. When labeling and learning information such as text content or pictures for large models, they were originally annotated through text, which resulted in low efficiency during labeling training. In addition, users cannot directly annotate on OFD files, resulting in a poor operating experience. Summary of the invention

[0004] The purpose of the present invention is to provide an OFD-based original text annotation machine learning method to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an OFD-based original text annotation machine learning method, characterized in that it comprises the following steps:

[0006] Step S1: Upload the file for training marking, determine its file format, and convert files of different formats into OFD files;

[0007] Step S2: Setting a multi-dimensional standard data element set, supporting multi-level cascading;

[0008] Step S3: Open the annotation reader, load the selected OFD file, render the file, and preload the set multi-level data element set in the memory;

[0009] Step S4: Open the OFD original text, select the content in the file in the OFD original text, match the corresponding data items, and record a series of content information such as the tag content, label data items, format, and location;

[0010] Step S5: Generate data from the recorded tag information and load it into the OFD extension file in XML format to generate an OFD file with a training model;

[0011] Step S6: Use a reader to open the OFD file, display the information of the annotated content, and click on the annotated elements to jump to the original text;

[0012] Step S7: Importing OFD files in batches into the training model to train the model files;

[0013] Step S8: Input the new OFD file into the model, automatically learn, extract the training element content, generate a new xml package, and integrate the annotated xml package into the OFD file.

[0014] Preferably, when judging the file format, if it is a picture or a scanned PDF file, the file is subjected to OCR recognition to generate a double-layer OFD file. If it is an audio or video file, the voice in the audio and video is converted into text and displayed synchronously below the audio and video. If it is a file in other formats, it is directly converted into an OFD file.

[0015] Preferably, the recorded mark information includes the marked text, label, marked position, and format, wherein the format includes: font, font size, color, and regular relationship.

[0016] Preferably, the generating of the OFD file with the training model comprises the following steps:

[0017] Step 1: Generate data from the recorded tag information and organize the generated data in XML format;

[0018] Step 2: Write the data in XML format to the corresponding OFD extension file to form an OFD file with a training model.

[0019] Preferably, the OFD file with the training model generated in step S5 is opened with a reader, and the file is displayed as it is, the corresponding annotated file content is selected, and the annotated elements are displayed;

[0020] The annotated elements are summarized to form an annotation overview table. When you click on an annotated element, you can jump to the original location.

[0021] Preferably, the training of the model file comprises the following steps:

[0022] Step 1: Import OFD files into the model in batches and parse the data in the OFD extension files;

[0023] Step 2: Create a training model and input the file into the model file for training.

[0024] Preferably, the training content of the file input into the model file for training includes text, location, and format.

[0025] Preferably, the OFD file integrated in step S8 is read, and when clicking on the elements extracted by the training model, it supports tracing back to the original source, supports visually viewing the detailed information of the marked file, and verifies the marked content in the file according to the training model.

[0026] A machine learning device for original text annotation across file formats, characterized by comprising a conversion module, a reading module, a annotation module, a file processing module, a training module, and a continuous improvement module;

[0027] The conversion module is used to determine the format of the file and the format conversion of the file;

[0028] The reading module supports displaying the original OFD file as well as displaying the annotated content;

[0029] The annotation module is used to provide multiple ways to select content in the reader and match the annotated elements;

[0030] The file processing module is used to generate an xml file from the trained file and store it in the OFD file to form a new OFD file;

[0031] The training module is used to parse the OFD file after importing it, read the training file, and import it into the model for training and learning;

[0032] The continuous improvement module not only extracts corresponding content and elements for the trained files, but also supports reverse derivation of marked content, further import and re-training.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The present invention uses OFD format as the unified format of knowledge base files, fully utilizes the functional advantages of OFD files, gives full play to the role of the xml file package of OFD files, and innovatively injects the training results into the xml file package of each OFD file. It can not only accurately bind the annotations to each file, but also use the structural rules of the OFD file xml to batch train other OFD files, greatly reducing the workload of user annotation;

[0035] (2) The present invention uses OFD format as the unified format of knowledge base files, which greatly expands the scope of data element sets, enables annotations to go beyond semantics, and expands the scope of fonts, font sizes, colors, regularization, structures, positions, formats, etc.;

[0036] (3) The present invention allows users to mark directly on OFD files, which improves the operating experience as the original version;

[0037] (4) The present invention enables the user to directly see the original OFD text when the AI ​​calculation results need to be traced back, making the result interpretation more intuitive and more credible. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The original text annotation machine learning device based on OFD consists of a conversion module, a reading module, an annotation module, a file processing module, a training module, and a continuous improvement module.

[0041] The conversion module is used to determine the format of the file and convert the format of the file;

[0042] The reading module supports the display of the original OFD file and the display of the annotated content;

[0043] The annotation module is used to provide multiple ways to select content in the reader and match the annotated elements;

[0044] The file processing module is used to generate an XML file from the trained file and store it in the OFD file to form a new OFD file;

[0045] The training module is used to parse the OFD file after importing it, read the training file, and import it into the model for training and learning;

[0046] The continuous improvement module not only extracts the corresponding content and elements for the trained files, but also supports reverse derivation of marked content, further import and re-training. Example

[0047] A machine learning method for text annotation based on OFD is implemented as follows:

[0048] Step S1: Upload the file for training marking, determine its file format, and convert files of different formats into OFD files;

[0049] When judging the file format, if the file format is a picture or a scanned PDF file, the file is subjected to OCR recognition to generate a double-layer OFD file. If the file format is an audio or video file, the voice in the audio or video is converted into text and displayed synchronously below the audio or video. If the file format is another format, it is directly converted into an OFD file.

[0050] Step S2: Setting a multi-dimensional standard data element set, supporting multi-level cascading;

[0051] Step S3: Open the annotation reader, load the selected OFD file, render the file, and preload the set multi-level data element set in the memory;

[0052] Step S4: Open the OFD original text, select the content in the file in the OFD original text, match the corresponding data items, and record a series of information such as the marked content, label data items, format, position, etc.; the recorded marked information includes the marked text, label, marked position, format, where the format includes: font, font size, color and regular relationship;

[0053] Step S5: Generate data from the recorded tag information, and organize the generated data in XML format;

[0054] Step S6: writing the data in XML format into the corresponding OFD extension file to form an OFD file with a training model;

[0055] Step S7: Use a reader to open the OFD file, display the information of the annotated content, and click on the annotated elements to jump to the original text;

[0056] Step S8: Batch import the OFD files into the model and parse the data in the OFD extension files;

[0057] Step S9: Create a training model, input the file into the model file for training, and the training content includes text, location, and format;

[0058] Step S10: input the new OFD file into the model, automatically learn, extract the training element content, generate a new xml package, integrate the annotated xml package into the OFD file, use the OFD format as the unified format of the knowledge base file, make full use of the functional advantages of the OFD file, give full play to the role of the xml file package of the OFD file, and innovatively inject the training results into the xml file package of each OFD file, which can not only accurately bind the annotation to each file, but also use the structural rules of the OFD file xml to batch train other OFD files, greatly reducing the workload of user annotation. Using the OFD format as the unified format of the knowledge base file greatly expands the scope of the data element set, so that the annotation goes beyond semantics and expands the scope of fonts, font sizes, colors, regular expressions, structures, positions, formats, etc.

[0059] Step S11: After the OFD file is integrated, when clicking on the elements extracted by the training model, it supports tracing back to the original source, supports visually viewing the detailed information of the marked file, and verifies the marked content in the file according to the training model;

[0060] Step S12: Generate an OFD file with a training model and open it with a reader. Display the file as it is. Select the corresponding annotated file content, display the annotated elements, summarize the annotated elements, and form an annotation overview table. When you click on the annotated element, you can jump to the original location.

[0061] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A machine learning method for text annotation based on OFD, It is characterized in that The following steps are involved: Step S1: Upload the file for training marking, determine its file format, and convert files of different formats into OFD files; Step S2: Setting a multi-level data element set with multi-dimensional standards, supporting multi-level cascading; Step S3: Open the annotation reader, load the selected OFD file, render the file, and preload the set multi-level data element set in the memory; Step S4: Open the OFD file, select the content in the file in the OFD file, match the corresponding data items, and record the information of the tag content, label data items, format, and location; Step S5: Generate data from the recorded tag information and load it into the OFD extension file in XML format to generate an OFD file with a training model; Step S6: Use a reader to open the OFD file, display the information of the annotated content, and click on the annotated elements to jump to the original text; Step S7: Importing OFD files in batches into the training model to train the model files; Step S8: Input the new OFD file into the training model, automatically learn, extract the training element content, generate a new xml package, and integrate the annotated xml package into the OFD file.

2. According to claim 1, a machine learning method for original text annotation based on OFD, Features: When judging the file format, if it is a picture or a scanned PDF file, the file is subjected to OCR recognition to generate a double-layer OFD file. If it is an audio or video file, the voice in the audio and video is converted into text and displayed synchronously below the audio and video. If it is a file in other formats, it is directly converted into an OFD file.

3. The machine learning method for text annotation based on OFD according to claim 1, Features: The information of the record mark includes the marked text, label, marked position and format, wherein the format includes: font, font size, color and regular relationship.

4. The machine learning method for text annotation based on OFD according to claim 1, Features: The generation of the OFD file with the training model comprises the following steps: Step 1: Generate data from the recorded tag information and organize the generated data in XML format; Step 2: Write the data in XML format to the corresponding OFD extension file to form an OFD file with a training model.

5. The machine learning method for text annotation based on OFD according to claim 1, Features: The OFD file with the training model generated in step S5 is opened with a reader and displayed as the original file, the corresponding annotated file content is selected, and the annotated elements are displayed; The annotated elements are summarized to form an annotation overview table. When you click on an annotated element, you can jump to the original location.

6. The machine learning method for text annotation based on OFD according to claim 1, Features: The training of the model file comprises the following steps: Step 1: Import OFD files into the training model in batches and parse the data in the OFD extension files; Step 2: Create a training model and input the file into the training model for training.

7. The machine learning method for text annotation based on OFD according to claim 6, Features: The training content of the file input into the model file for training includes text, position, and format.

8. The machine learning method for text annotation based on OFD according to claim 1, Features: Read the OFD file integrated in step S8, and when clicking on the elements extracted by the training model, you can trace back to the original source, visualize the detailed information of the marked file, and verify the marked content in the file according to the training model.

9. A machine learning device for text annotation based on OFD, Features: It includes conversion module, reading module, annotation module, document processing module, training module and continuous improvement module; The conversion module is used to determine the format of the file and the format conversion of the file; The reading module supports displaying the original OFD file as well as displaying the annotated content; The annotation module is used to provide multiple ways to select content in the reader and match the annotated elements; The file processing module is used to generate an xml file from the trained file and store it in the OFD file to form a new OFD file; The training module is used to parse the OFD file after importing it, read the training file, and import it into the model for training and learning; The continuous improvement module not only extracts corresponding content and elements for the trained files, but also supports reverse derivation of marked content, further import and re-training.

Citation Information

Patent Citations

  • Method for merging and displaying electronic form and OFD format file and generating catalogue

    CN111753499A

  • Method and device for combining multi-format files into OFD file

    CN113961531A

  • Intelligent labeling and extracting method for PDF (Portable Document Format) document

    CN116070602A

  • Open format document opening method and device, electronic equipment and storage medium

    CN116340271A

  • Model training method, standard format document generation method and device

    CN116484223A