A method, device and computing equipment for processing format files

Abstract

The present invention provides a method, device and computing equipment for processing layout files. The method for processing layout files includes: obtaining preset type file data; converting the preset type file data into a standard layout file; extracting at least one cover data of the standard layout file to obtain first extracted data; extracting metadata of the preset type file data to obtain second extracted data; extracting hierarchical data of the preset type file data to obtain third extracted data; embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file. The present invention can meet the requirements of data archiving in different scenarios and improve the efficiency and convenience of using layout files.

Description

Technical Field

[0001] The present invention relates to the technical field of format file processing, and in particular to a format file processing method, device and computing equipment. Background Art

[0002] OFD is a standard format for electronic document exchange and storage, and is also an autonomous and controllable electronic document format. OFD format files have a fixed layout, no layout changes, and what you see is what you get. They can be regarded as the "digital paper" of the computer age. They are an ideal document format for electronic document publishing, digital information dissemination, and archiving. OFD format has obvious advantages in electronic document archiving management. However, the existing OFD format files cannot meet the requirements of data archiving in different scenarios. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method, device and computing equipment for processing layout files, which can meet the requirements of data archiving in different scenarios and improve the efficiency and convenience of using layout files.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A method for processing a layout file, comprising:

[0006] Get the preset type file data;

[0007] Converting the preset type file data into a standard format file;

[0008] Extracting at least one cover data of the standard layout file to obtain first extracted data;

[0009] Extracting metadata of the preset type of file data to obtain second extracted data;

[0010] Extracting hierarchical data of the preset type of file data to obtain third extracted data;

[0011] The first extracted data, the second extracted data and the third extracted data are embedded into the standard layout file to obtain a target layout file.

[0012] Optionally, converting the preset type file data into a standard format file includes:

[0013] Parsing the file type of the preset type file data;

[0014] Call the standard layout file template;

[0015] The data packet of the file data of the file type is converted according to the standard layout file template to obtain a standard layout file.

[0016] Optionally, extracting at least one cover data of the standard layout file to obtain first extracted data includes:

[0017] Parsing the color and transparency data of the standard layout file to obtain a parsing result;

[0018] In the analysis result, the first page image of the standard layout file is rendered to obtain cover data, and the cover data is used as the first extracted data.

[0019] Optionally, extracting metadata of the preset type of file data to obtain second extracted data includes:

[0020] Converting the preset type file data into text data;

[0021] Semantic recognition is performed on the text data to obtain multiple metadata, and a set of the multiple metadata is used as the second extracted data.

[0022] Optionally, extracting the hierarchical data of the preset type of file data to obtain third extracted data includes:

[0023] Extracting classification data, first-level data and second-level data from the preset type file data;

[0024] The third extracted data is obtained according to the classification data, the first-level data and the second-level data; the first-level data includes: entity data, and the second-level data includes: abstract, keyword and knowledge triple data.

[0025] Optionally, extracting the classification data from the preset type file data includes:

[0026] Masking and replacing a predetermined proportion of words in a preset type of file data by using a generator of a preset training language model;

[0027] Label the replacement and non-replacement words respectively to get the replacement text;

[0028] Performing replacement word detection on the replacement text by using the discriminator of the preset training language model, and obtaining an information extraction model after fine-tuning;

[0029] The classification data is extracted using the classifier of the information extraction model.

[0030] Optionally, embedding the first extracted data, the second extracted data, and the third extracted data into the standard layout file to obtain a target layout file includes:

[0031] According to the format of a standard layout file, converting the first extracted data, the second extracted data and the third extracted data into objects of the standard layout file respectively;

[0032] The objects of the standard layout file are respectively added to the metadata nodes in the standard layout file to obtain the target layout file.

[0033] An embodiment of the present invention further provides a format file processing device, comprising:

[0034] An acquisition module is used to acquire file data of a preset type;

[0035] A processing module is used to convert the preset type file data into a standard layout file; extract at least one cover data of the standard layout file to obtain first extracted data; extract metadata of the preset type file data to obtain second extracted data; extract hierarchical data of the preset type file data to obtain third extracted data; embed the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file.

[0036] An embodiment of the present invention further provides a computing device, comprising:

[0037] one or more processors;

[0038] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the format file processing method described in the present invention.

[0039] An embodiment of the present invention further provides a computing device readable storage medium, wherein the computing device readable storage medium stores a program, and when the program is executed by a processor, the method for processing a layout file of the present invention is implemented.

[0040] The above technical solution of the present invention has at least the following technical effects:

[0041] The above-mentioned layout file processing method of the present invention obtains preset type file data; converts the preset type file data into a standard layout file; extracts at least one cover data of the standard layout file to obtain first extracted data; extracts metadata of the preset type file data to obtain second extracted data; extracts hierarchical data of the preset type file data to obtain third extracted data; embeds the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file. The requirements for data archiving in different scenarios are met, and the efficiency and convenience of using layout files are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the method for processing a format file of the present invention;

[0043] Figure 2 is a schematic diagram of a BERT model of a method for processing a layout file of the present invention;

[0044] Figure 3 is a schematic diagram of a Transformers encoder of the method for processing a layout file of the present invention;

[0045] Figure 4 It is a schematic diagram of the ELECTRA model of the method for processing the layout file of the present invention;

[0046] Figure 5 It is a data extraction schematic diagram of the method for processing a layout file of the present invention;

[0047] Figure 6 It is a schematic diagram of the format file processing device of the present invention. DETAILED DESCRIPTION

[0048] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a method for processing a layout file, comprising:

[0050] Step S1, obtaining file data of a preset type;

[0051] Step S2, converting the preset type file data into a standard format file;

[0052] Step S3, extracting at least one cover data of the standard layout file to obtain first extracted data;

[0053] Step S4, extracting metadata of the preset type of file data to obtain second extracted data;

[0054] Step S5, extracting the hierarchical data of the preset type of file data to obtain third extracted data;

[0055] Step S6, embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file.

[0056] In the above scheme of the present invention, the layout file (OFD, Open Fixed-layout Document) is a standard format for electronic document exchange and storage, and the OFD format is an autonomous and controllable layout file format.

[0057] In this embodiment, Figure 1 As shown, firstly, the preset type file data is converted into a standard layout file, so that the file format is changed and a standard layout file structure framework is built at the same time, and then the automatic extraction technology is used to extract the cover data, metadata and hierarchical data respectively, and finally the cover data, metadata and hierarchical data are embedded in the standard layout file to obtain the target layout file.

[0058] The solution of the present invention can automatically extract various embedded data required to supplement the OFD file, and obtain an OFD standard file with complete data elements and rich content, thereby improving the efficiency and convenience of using the OFD standard file and meeting the requirements of data archiving in different scenarios.

[0059] In an optional embodiment of the present invention, in step S2, converting the preset type file data into a standard format file may include:

[0060] Step 21, parsing the file type of the preset type file data;

[0061] Step 22, calling a standard layout file template;

[0062] Step 23, converting the data packet of the file data of the file type according to the standard layout file template to obtain a standard layout file.

[0063] In this embodiment, specifically, parsing the file type of the preset type file data in step 21 may include: verifying the integrity of the preset type file data, using the MD5 value (message digest algorithm) to verify the file integrity, for example: the preset type file data is named as the file type + file MD5 value (such as: policy file-b9833c3526c4e578.zip), after the server reads the file information, it uses the digest method in the MessageDigest class to calculate the MD5 value of the file content, and confirms the integrity of the file content by comparing the calculation result with the incoming file name; calling the digital processing software of the preset type file data, parsing the file type of the preset type file data, analyzing the elements of the file, identifying the file type, and if the file type is not an OFD file, converting its format.

[0064] In step 23, converting the data packet of the file data of the file type according to the standard layout file template to obtain a standard layout file may include:

[0065] Step 231, converting the preset type file data into a document model;

[0066] Step 232, editing the document model into view information;

[0067] Step 233, performing layout conversion on the view information to obtain a layout document;

[0068] Step 234, processing the layout document to generate the standard layout file.

[0069] First, convert the preset type file data into a document model; then, call the typesetting processing software, and edit the document model into view information that can be displayed and printed according to the OFD typesetting algorithm, and keep the original format of the preset type file data; secondly, according to the OFD layout conversion rules, analyze and sort out the view information of the document model, and perform layout conversion; finally, according to the element composition requirements of the OFD layout file template, establish the layout file framework to obtain the initial standard layout file.

[0070] In an optional embodiment of the present invention, in step S3, extracting at least one cover data of the standard layout file to obtain first extracted data may include:

[0071] Step 31, parsing the color and transparency data of the standard layout file to obtain a parsing result;

[0072] Step 32, in the analysis result, render the first page image of the standard layout file to obtain cover data, and use the cover data as the first extracted data.

[0073] In this embodiment, specifically, digital processing software for preset type file data is called to parse information such as color and transparency data of the standard layout file to obtain a parsing result; based on the parsing result, the first page image of the standard layout file is rendered to obtain cover data. The OFD layout file can be toned by setting the color within the channel value color gamut, or the corresponding color can be used and configured through a custom color configuration file, which supports RGB, Gray, CMYK and other color spaces; the obtained cover data is stored as the first extracted data in a data server.

[0074] In an optional embodiment of the present invention, in step S4, extracting metadata of the preset type of file data to obtain second extracted data may include:

[0075] Step 41, converting the preset type file data into text data;

[0076] Step 42: perform semantic recognition on the text data to obtain multiple metadata, and use a collection of the multiple metadata as the second extracted data.

[0077] In this embodiment, specifically, the optical character recognition function is used to identify the preset type of file data as text data; in the text data, the metadata is extracted to obtain the second extracted data; for example, the metadata of an official document includes: official document identification, document type, copy number, confidentiality level and confidentiality period, urgency, issuing agency mark, document number, issuer, title, main addressee, attachment description, signature of the issuing agency or issuer, date of writing, notes, copy agency, printing and issuing agency, printing and issuing date, and release level.

[0078] In an optional embodiment of the present invention, in step S5, extracting the hierarchical data of the preset type file data to obtain the third extracted data may include:

[0079] Step 51, extracting the classification data, the first level data and the second level data from the preset type file data;

[0080] Step 52, obtaining third extracted data according to the classification data, the first-level data and the second-level data; the first-level data includes: entity data, and the second-level data includes: abstract, keyword and knowledge triple data.

[0081] In an optional embodiment of the present invention, in step 51, extracting classification data from the preset type file data may include:

[0082] Step 511, masking and replacing a predetermined proportion of words in the preset type of file data by using a generator of a preset training language model;

[0083] Step 512, label the replacement and non-replacement characters respectively to obtain replacement text;

[0084] Step 513, performing replacement word detection on the replacement text through a discriminator of a preset training language model, and obtaining an information extraction model after fine-tuning;

[0085] Step 514: Use the classifier of the information extraction model to extract and obtain classified data.

[0086] Specifically, the preset training language model in this embodiment can use a BERT (Bidirectional Encoder Representations from Transformers) pre-trained model to extract classified data; in specific implementation, such as Figure 2 As shown in the figure, firstly, multiple or multiple pairs of sentences in the preset type of file data (such as questions and answers in the question-answering system) are input into the BERT model. The generator of the BERT model uses three vectors, word embedding, segment embedding, and position embedding, to represent the words in each input sentence, and then uses Figure 3 The encoder of the Transformers model shown in the figure processes the input representation. Each layer of the Transformers encoder contains a self-attention mechanism and a feed-forward neural network, and is connected through residual connections and layer normalization. Normalization); secondly, the BERT model is trained for language learning representation using two pre-training tasks, the masked language model (MLM) and the next sentence prediction (NSP). The masked language model randomly masks some words in the input, and then requires the model to predict these masked words and learn the relationship between words in the context; for the next sentence prediction (NSP), given a pair of sentences, the model needs to predict whether the second sentence is the true next sentence of the first sentence, so as to determine the relationship between the sentences; after the pre-training is completed, the BERT model is fine-tuned according to a specific classification data extraction task. In the fine-tuning stage, an output layer for the classification data extraction task (such as a classification layer or a sequence annotation layer) is added, and then the model is trained using the labeled data. The fine-tuning process adjusts all parameters (including the parameters in the pre-training stage) to adapt to the classification data extraction task; finally, the trained BERT model is the information extraction model, and the information extraction model is used to extract classification data to obtain classification data.

[0087] In an optional embodiment of the present invention, in step 51, extracting the first-level data and the second-level data in the preset type file data may include:

[0088] Step 515, masking and replacing the words in the preset type of file data by using a generator of a preset network learning model;

[0089] Step 516, label the replaced and non-replaced words respectively to obtain the replaced text;

[0090] Step 517, performing replacement word detection on the replacement text through a discriminator of a preset network learning model, and obtaining an information extraction model after fine-tuning;

[0091] Step 518: Use the classifier of the information extraction model to extract the first-level data and the second-level data.

[0092] In this embodiment, Figure 4 As shown, the preset network learning model can use the ELECTRA model (Efficiently Learning an Encoder that Classifies Token Replacements Accurately) to extract the first-level data and the second-level data;

[0093] First, the ELECTRA model is pre-trained using a generator and a discriminator. The generator is a masked language model (MLM), which randomly selects a portion of words in the input text to mask (usually 15% of the words) and tries to predict these masked words. In this way, the generator learns the contextual representation of the language and generates reasonable alternative words to replace the words in the masked position.

[0094] Then, the text processed by the generator, that is, the text in which some words are replaced by the replacement words predicted by the generator, is input into the discriminator. The discriminator will determine whether each word in the input text is replaced, that is, perform the replacement word detection task. By learning to distinguish between the original words and the words predicted by the generator, the model's ability to understand the language structure is improved;

[0095] After pre-training is completed, the ELECTRA model is fine-tuned according to the first-level data and second-level data extraction tasks. The fine-tuning adopts the low-rank adaptation (LoRA) method. The LoRA method fine-tunes the model by injecting a trainable low-rank decomposition matrix, thereby greatly reducing the number of trainable parameters required for downstream tasks. During the fine-tuning process, the original pre-trained weights of the ELECTRA model are first frozen so that these weights will not change during the training process, and then the trainable low-rank decomposition matrix is ​​inserted at the appropriate position of the discriminator. Secondly, the inserted low-rank matrix is ​​trained, and the parameters of these matrices are updated by minimizing the objective function of the downstream task; finally, the preset type file data is input into the trained ELECTRA model, that is, the information extraction model, and the words are converted into vector representations through the information extraction model. Then, the classifier of the information extraction model is used to identify the first-level data and the second-level data in the extracted text.

[0096] In an optional embodiment of the present invention, in step S6, embedding the first extracted data, the second extracted data, and the third extracted data into the standard layout file to obtain the target layout file may include:

[0097] Step 61, converting the first extracted data, the second extracted data and the third extracted data into objects of the standard layout file respectively according to the format of the standard layout file;

[0098] Step 62, respectively add the objects of the standard layout file to the metadata nodes in the standard layout file to obtain the target layout file.

[0099] In this embodiment, specifically, the OFD software development kit is first used to open the established standard layout file, and then the first extraction data, the second extraction data and the third extraction data are obtained. Secondly, the OFD software development kit is used to embed the first extraction data, the second extraction data and the third extraction data into the standard layout file, and finally the embedding result is saved to make the first extraction data, the second extraction data and the third extraction data become part of the standard layout file, thereby obtaining the target layout file.

[0100] The following takes an official document as an example to illustrate the specific implementation process of the above method of the present invention:

[0101] like Figure 5 As shown, the present invention embeds the file information (file basic information, metadata information, etc.) into the standard OFD file, and the specific steps are:

[0102] 1. Data import includes file data, basic file information (file size, file name, file format, file summary MD5 value, file creation time), and metadata information (industry metadata, intelligent metadata, and customer-defined metadata database information);

[0103] 2. The data is cleaned according to the defined metadata standard data format and range, and the metadata is manually checked, and then written into the standard OFD file;

[0104] 3. Data extraction includes cover, classification, abstract, keyword, entity, and knowledge triple information;

[0105] 4. Data archive contains file, metadata, cover, classification, summary, keyword, entity, and knowledge triple information. It is embedded in OFD file;

[0106] Specifically:

[0107] The data import stage includes source files, attachments, basic file information (file size, file name, file format, file summary MD5 value, file creation time), metadata information (industry metadata, intelligent metadata, customer-defined metadata database information), and other information.

[0108] 1) Official document industry data import stage:

[0109] 1.1 In order to verify the integrity of the file package size, the MD5 value of the file package content is used to verify the file integrity, for example:

[0110] Input: file package name file type + file MD5 value (such as: policy file-b9833c3526c4e578.zip)

[0111] Output: After the server reads the file information, it uses the digest method in the MessageDigest class to calculate the MD5 value of the file content and compares the calculation result with the passed-in file name to confirm the integrity of the file content.

[0112] 1.2 Store the third-party attached file information and metadata information in the server database, for example:

[0113] Input: three-party document ID, full name of the main document, document identification, document type, copy number, confidentiality level and confidentiality period, urgency, issuing agency mark, document number, issuer, title, main recipient agency, attachment description, signature of the issuing agency or issuer, date of writing, notes, copy recipient agency, issuing agency, date of issuance, release level, full name of attachment file, handling metadata, permission set, classification set, main document AI metadata, extended metadata and other information.

[0114] Output: Store the corresponding file information into the server database.

[0115] The attached metadata information is as follows:

[0116] parameter type Is it necessary illustrate --metadata Object yes The metadata of official documents refers to the metadata description of official documents. The metadata attribute name can be defined by the user, but the mapping relationship must be synchronized with the official document database. ---fileType String yes File type value such as: 000 policy file connection specific confirmation with the project ---storeLocation list <string> < / string> yes Storage location collection ---id String yes The third-party document ID cannot be empty (the maximum length is 32 characters, and can also be 19 characters) ---fileName String yes Full name of the subject document ---filePath String no Main document path ---gwbs String no Official document identification ---wz String yes Type of writing ---fh String no Copy number ---mjqx String yes Classification and confidentiality period ---jjcd String no Urgency ---fwjgbz String yes Issuing agency logo ---fwzh String yes Document number ---qfr String no Issuer ---bt String no title ---zsjg String no Main sending agency ---fjsm String no Attachment Description ---fwjgsm String no Signature of issuing authority or issuer --cwrq String yes Date of writing ---fz String no Notes ---csjg String no Copy to ---yfjg String no Issuing agency ---yfrq String no Date of issue ---fbcc String no Release level --attachments Object[] no Attachment file list information ---name String no Attachment file full name ---path String no Attachment path --processingMetadata Object[] no If the metadata is set for approval, the attributes in the corresponding array must be filled in, and multiple link nodes are set in the array in sequence according to the process order. ---content String no Content of opinion ---approver String no Approver ---approvalTime String no Approval Time ---approvalDept String no Approval department ---processNode String no Process link (node) --permissionSet Object no Permission Set ---rangeType Integer no The document disclosure range type is 0-no additional specification, 1-all disclosure, 2-specified range, the default is 0, when the value is 0 or 1, rangeDetails is not required ---rangeDetails Object[] no Visibility details ----orgType String no Visible scope specific categories dept-specified department, user-specified user, role-specified role, specific categories can exist at the same time or separately ----orgIds String no The id corresponding to the specific category, such as department id, user id, etc., and so on ----permissionType String no Consultation on user ID validity --classificationSet Set <string> < / string> no Classification set, fill in the Chinese value, use / to separate the levels, the beginning is the root category, example: theme / comprehensive government affairs / government affairs disclosure single example: economic development --aiMetadata Object no Master AI metadata ---keywords Set <string> < / string> no Keywords ---dateList Set <string> < / string> no Entity Date ---locationList Set <string> < / string> no Physical location ---personList Set <string> < / string> no Entity Character ---organizationList Set <string> < / string> no Physical Institutions --extMetadata Map<String,Object> no Extended metadata (key: Chinese name, value: custom)

[0117] 1.3 Data import phase When the interface receives the above information, it determines the main file format type. If it is a file type other than OFD, it is necessary to use the conversion service to convert files of other formats into standard OFD files. The usage method is to embed the SDK package of the conversion service, for example:

[0118] Input: file, package type, target format type.

[0119] Output: Use the standard ofd standard to parse the input file into an input object, convert the object into an ofd file object according to the standard ofd standard, and output the ofd file.

[0120] Specifically, Packet pa = new Packet(Const.PackType.COMMON, Const.Target.OFD); then use the convertNoWait(pa, convertCallback) method to obtain the standard ofd file, and provide a callback method to obtain the conversion life cycle.

[0121] 2) Data extraction: cover, metadata, classification, abstract, keyword, entity, and knowledge triple information.

[0122] 2.1 Data extraction includes using the API5 SDK toolkit to extract the cover image, for example:

[0123] Input: file data, set color, set transparency, output format, output path.

[0124] Output: parse the ofd document and render the first page, and generate a cover image under the output path.

[0125] Specifically:

[0126] / / Create SDK api instance

[0127] API api = API.create();

[0128] Open open = new Open(Data.ofMem(ofdFileByte));

[0129] / / Open the API instance and pass in the file data

[0130] api.call(open);

[0131] / / Set the cover image format RenderPage renderPage = new RenderPage(newIndexBaseOne(1));

[0132] renderPage.setDIBFormat(RenderPage.DIBFormat.BGR32);

[0133] renderPage.setImageType(RenderPage.ImageType.PNG);

[0134] renderPage.setTargetFilePath(imgPath);

[0135] / / Extracted cover image

[0136] api.call(renderPage);

[0137] / / Close the API instance

[0138] api.call(new Close());

[0139] Finally get the cover image extracted by api5.

[0140] 2.2 Data extraction includes using OCR services to extract 18 metadata items of official documents using http, such as:

[0141] Input: file data.

[0142] Output: Recognize the scanned file into text, and use the official document semantic recognition algorithm to extract the official document metadata, 18 items of official document metadata (official document identification, document type, copy number, confidentiality level and confidentiality period, urgency, issuing agency mark, document number, issuer, title, main addressee, attachment description, signature of the issuing agency or issuer, date of writing, notes, copy agency, issuing agency, issuance date, and release level).

[0143] Specifically:

[0144] / / Construct request body

[0145] RequestBody requestBody = RequestBody.create(MediaType.parse(CommonService.MEDIATYPE_FORMDATA), file);

[0146] / / Construct file upload object

[0147] MultipartBody.Part part = MultipartBody.Part.createFormData("image_file", file.getName(), requestBody);

[0148] / / Request callback path parameters

[0149] RequestBody param1 = RequestBody.create(MediaType.parse("text / plain"), callBackUrl);

[0150] / / Request extended parameters

[0151] RequestBody param2 = RequestBody.create(MediaType.parse("text / plain"), extendParam);

[0152] / / Request form parameters

[0153] RequestBody formatReq = RequestBody.create(MediaType.parse("text / plain"), format);

[0154] / / Request address

[0155] String allUrl= url + " / classify / async";

[0156] / / Call OCR service

[0157] classifyWithUrl(allUrl, part,param1,param2,formatReq);

[0158] Finally, 18 metadata items of official documents were obtained.

[0159] 2.3 Data extraction includes using AI services to extract classification, summary, keyword, entity, and knowledge triple information in a message queue manner, for example:

[0160] 2.3.1 File Classification

[0161] Input: file data.

[0162] Output: Convert the hierarchical classification of the file into multi-label classification, use the BERT pre-trained model for feature extraction, and use sigmoid cross entropy loss for classification.

[0163] 2.3.2 Entity Extraction, Summary, Keyword, and Knowledge Triple Extraction

[0164] Input: file data.

[0165] Output: Entity, summary, keyword, knowledge triple extraction is based on the ELECTRA model. The ELECTRA model is mainly composed of two parts: the generator and the discriminator. First, the generator masks and replaces 15% of the words in the input text, and labels the replaced and non-replaced words respectively; then the discriminator determines whether the output of the generator has been replaced. Based on the data, the open source large model is efficiently fine-tuned by LoRa. During the training process, LoRa drops normally. During the inference decoding, a combination of multiple decoding strategies is used to generate summary, keyword, and knowledge triples.

[0166] Specifically, the file information is pushed to the Ai middle station through the message queue.

[0167] After receiving the message, the Ai middle station calls the algorithm platform to extract the corresponding classification, summary, keyword, entity, and knowledge triple information in the text.

[0168] Obtain the corresponding classification, summary, keyword, entity, and knowledge triple information through callback.

[0169] 3) Data archiving includes files, metadata, cover, classification, summary, keywords, entities, and knowledge triples. Information conversion service interface is embedded in OFD file, for example:

[0170] Input: 18 metadata items of files and official documents (official document identification, document type, copy number, confidentiality level and confidentiality period, urgency, issuing agency mark, document number, issuer, title, main recipient agency, attachment description, signature of the issuing agency or issuer, date of writing, notes, copy recipient agency, issuing agency, date of issuance, and release level), cover image, classification, abstract, keyword, entity, and knowledge triplet information.

[0171] Output: Parse into OFD object according to OFD standard and add corresponding metadata file to metadata node.

[0172] Specifically, the input parameters include:

[0173] parameter type Is it necessary illustrate Example components Array yes A collection of documents to be converted (with metadata added). If there are multiple documents, the document will be merged and converted. [{"fileLoc": "http: / / 127.0.0.1:8090 / file / 2.ofd","format": "ofd"}] components.fileLoc String yes Data url, must be a standard uri path "http: / / 127.0.0.1:8090 / file / 2.ofd""file: / / / D: / 1.ofd"Here is the server file address. It is recommended to use the remote file url method above to pass parameters. components.format String yes Data Format "ofd" components.title String no Document Title "title" metaRoot Object yes Metadata Components {"docID":"123456","title": "test"} metaRoot.docID String no Document id "123456" metaRoot.title String no Document Title "test" metaRoot.author String no Document Author "user" metaRoot.creator String no Document creator "user" metaRoot.customDatas Array no Custom metadata components [{"name":"custom key","value":"custom value"}] metaRoot.customDatas.name String no Custom metadata key "Custom key" metaRoot.customDatas.value String no Custom metadata value "Custom value"

[0174] The above-mentioned embodiment of the present invention embeds various metadata into the OFD file, provides data support for the identification and archiving links, organization and storage stage, retrieval and utilization stage, and offline management of electronic files, and ensures the authenticity and reliability of the data.

[0175] like Figure 6 As shown, in an embodiment of the present invention, a layout file processing device 60 is also provided, comprising:

[0176] An acquisition module 61 is used to acquire file data of a preset type;

[0177] Processing module 62 is used to convert the preset type file data into a standard layout file; extract at least one cover data of the standard layout file to obtain first extracted data; extract metadata of the preset type file data to obtain second extracted data; extract hierarchical data of the preset type file data to obtain third extracted data; embed the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file.

[0178] Optionally, converting the preset type file data into a standard format file includes:

[0179] Parsing the file type of the preset type file data;

[0180] Call the standard layout file template;

[0181] The data packets of the file data of the file type are converted according to the standard layout file template to obtain a standard layout file.

[0182] Optionally, extracting at least one cover data of the standard layout file to obtain first extracted data includes:

[0183] Parsing the color and transparency data of the standard layout file to obtain a parsing result;

[0184] In the analysis result, the first page image of the standard layout file is rendered to obtain cover data, and the cover data is used as the first extracted data.

[0185] Optionally, extracting metadata of the preset type of file data to obtain second extracted data includes:

[0186] Converting the preset type file data into text data;

[0187] Semantic recognition is performed on the text data to obtain multiple metadata, and a set of the multiple metadata is used as the second extracted data.

[0188] Optionally, extracting the hierarchical data of the preset type of file data to obtain third extracted data includes:

[0189] Extracting classification data, first-level data and second-level data from the preset type file data;

[0190] The third extracted data is obtained according to the classification data, the first-level data and the second-level data; the first-level data includes: entity data, and the second-level data includes: abstract, keyword and knowledge triple data.

[0191] Optionally, extracting the classification data from the preset type file data includes:

[0192] Masking and replacing a predetermined proportion of words in a preset type of file data by using a generator of a preset training language model;

[0193] Label the replacement and non-replacement words respectively to get the replacement text;

[0194] Performing replacement word detection on the replacement text by using the discriminator of the preset training language model, and obtaining an information extraction model after fine-tuning;

[0195] The classification data is extracted using the classifier of the information extraction model.

[0196] Optionally, embedding the first extracted data, the second extracted data, and the third extracted data into the standard layout file to obtain a target layout file includes:

[0197] According to the format of a standard layout file, converting the first extracted data, the second extracted data and the third extracted data into objects of the standard layout file respectively;

[0198] The objects of the standard layout file are respectively added to the metadata nodes in the standard layout file to obtain the target layout file.

[0199] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method are applicable to the embodiments of the device and can achieve the same technical effects.

[0200] The embodiment of the present invention further provides a computing device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the processing method of the layout file of the present invention. All implementation methods in the above method embodiment are applicable to the embodiment of the device, and can also achieve the same technical effect.

[0201] In an embodiment of the present invention, a computing device readable storage medium is further provided, including: a program is stored in the computing device readable storage medium, and when the program is executed by a processor, the method for processing a layout file of the present invention is implemented. All implementations in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0202] The solution of the present invention extracts information from the document content and embeds it into the OFD template to form a layout file, which is convenient for providing data support for subsequent identification and archiving links, sorting and storage stages, retrieval and utilization stages, offline management, etc., to ensure the authenticity and reliability of the data.

[0203] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing a layout file, characterized in that: include: Get the preset type file data; Converting the preset type file data into a standard format file; Extracting at least one cover data of the standard layout file to obtain first extracted data; Extracting metadata of the preset type of file data to obtain second extracted data; Extracting hierarchical data of the preset type of file data to obtain third extracted data; Embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file; The step of extracting the hierarchical data of the preset type of file data to obtain third extracted data includes: Extracting classification data, first-level data and second-level data from the preset type file data; According to the classification data, the first-level data and the second-level data, third extracted data is obtained; the first-level data includes: entity data, and the second-level data includes: abstract, keyword and knowledge triple data; The step of extracting the classification data from the preset type file data includes: Masking and replacing a predetermined proportion of words in a preset type of file data by using a generator of a preset training language model; Label the replacement and non-replacement words respectively to get the replacement text; Performing replacement word detection on the replacement text by using the discriminator of the preset training language model, and obtaining an information extraction model after fine-tuning; extracting classified data using a classifier of the information extraction model; The step of embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file includes: According to the format of a standard layout file, converting the first extracted data, the second extracted data and the third extracted data into objects of the standard layout file respectively; The objects of the standard layout file are respectively added to the metadata nodes in the standard layout file to obtain the target layout file.

2. The method for processing a layout file according to claim 1, characterized in that: The step of converting the preset type file data into a standard format file includes: Parsing the file type of the preset type file data; Call the standard layout file template; The data packet of the file data of the file type is converted according to the standard layout file template to obtain a standard layout file.

3. The method for processing a layout file according to claim 1, characterized in that: The step of extracting at least one cover data of the standard layout file to obtain first extracted data includes: Parsing the color and transparency data of the standard layout file to obtain a parsing result; In the analysis result, the first page image of the standard layout file is rendered to obtain cover data, and the cover data is used as the first extracted data.

4. The method for processing a layout file according to claim 1, characterized in that: The step of extracting metadata of the preset type of file data to obtain second extracted data includes: Converting the preset type file data into text data; Semantic recognition is performed on the text data to obtain multiple metadata, and a set of the multiple metadata is used as the second extracted data.

5. A layout file processing device, characterized in that: include: An acquisition module is used to acquire file data of a preset type; A processing module, used for converting the preset type file data into a standard format file; Extracting at least one cover data of the standard layout file to obtain first extracted data; extracting metadata of the preset type file data to obtain second extracted data; extracting hierarchical data of the preset type file data to obtain third extracted data; embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file; The step of extracting the hierarchical data of the preset type of file data to obtain third extracted data includes: Extracting classification data, first-level data and second-level data from the preset type file data; According to the classification data, the first-level data and the second-level data, third extracted data is obtained; the first-level data includes: entity data, and the second-level data includes: abstract, keyword and knowledge triple data; The step of extracting the classification data from the preset type file data includes: Masking and replacing a predetermined proportion of words in a preset type of file data by using a generator of a preset training language model; Label the replacement and non-replacement words respectively to get the replacement text; Performing replacement word detection on the replacement text by using the discriminator of the preset training language model, and obtaining an information extraction model after fine-tuning; extracting classified data using a classifier of the information extraction model; The step of embedding the first extracted data, the second extracted data and the third extracted data into the standard layout file to obtain a target layout file includes: According to the format of a standard layout file, converting the first extracted data, the second extracted data and the third extracted data into objects of the standard layout file respectively; The objects of the standard layout file are respectively added to the metadata nodes in the standard layout file to obtain the target layout file.

6. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computing device readable storage medium, characterized in that: The computing device readable storage medium stores a program, which implements the method according to any one of claims 1 to 4 when executed by a processor.

Images