Word document automatic indexing method and device based on short text classification task

Abstract

The application discloses a word document automatic indexing method and device based on a short text classification task, an electronic equipment and a storage medium. In the method, the word document automatic indexing task is converted into a short text classification task in the field of natural language processing. The task uses a pre-training language model for semantic understanding, can automatically mine potential features of data, and fully utilizes semantic features of articles in sample documents. Through a sample document structure feature and a first short text content generation algorithm, the structure features of the articles in the sample documents are converted into semantic features, so that the pre-training language model can consider the structure features of the articles while learning the semantic features of the sample paragraphs. The preset deep learning model obtained through training automatically establishes a strong dependency relationship between the paragraphs and the labels, so that better indexing effect is achieved.

Description

Technical Field

[0001] This application relates to the technical field of Word document indexing, and in particular to a method, apparatus, electronic device and storage medium for automatic indexing of Word documents based on short text classification tasks. Background Technology

[0002] Automatic indexing of Word documents involves tagging paragraphs within the document. This data processing facilitates information storage and retrieval, and forms the foundation for data resource integration and in-depth knowledge mining. For example, if the content of a paragraph contains keywords, that paragraph will be tagged, and the tags can specifically be keywords.

[0003] Currently, most traditional automatic indexing algorithms are rule-based, requiring the development of different rule-based indexing programs for documents with different styles and structures, resulting in low development efficiency. In addition, rule conflicts may occur for documents with relatively complex structures, limiting the scope of application. Furthermore, traditional rule-based automatic indexing algorithms face significant optimization bottlenecks, making it difficult to improve their performance. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for automatic indexing of Word documents based on a short text classification task. The method transforms the automatic indexing task of Word documents into a short text classification task within the field of natural language processing. This task utilizes a pre-trained language model oriented towards semantic understanding, which can automatically mine the latent features of the data and fully leverage the semantic features of the articles in the sample documents. Through the structural features of the sample documents and a first short text content generation algorithm, the structural features of the articles in the sample documents are transformed into semantic features. This allows the pre-trained language model to learn the semantic features of the sample paragraphs while also considering the structural features of the articles. The trained, pre-defined deep learning model automatically establishes strong dependencies between paragraphs and tags to achieve better indexing results.

[0005] In a first aspect, embodiments of this application provide a method for automatic indexing of Word documents based on a short text classification task, including:

[0006] Obtain a large number of sample documents, wherein sample paragraphs of the sample documents are marked with sample tags;

[0007] For the sample paragraphs in the sample document, extract the first basic feature after normalization;

[0008] The first basic feature, after being normalized, is processed to generate semantic sample document structure features;

[0009] The content of the first short text is determined based on the first basic feature of the sample paragraph and the structural feature of the sample document;

[0010] The first short text content is determined to be sample input data, and the sample label is determined to be a real label. The sample input data and the real label are used to form target sample data.

[0011] The pre-trained language model is trained using the target sample data to obtain a preset deep learning model;

[0012] Retrieve the Word document to be indexed;

[0013] For the document paragraphs in the Word document to be indexed, extract the basic features of the Word document after normalization, wherein the document paragraphs have preset features;

[0014] The basic features of the normalized Word document are processed to generate semantic target document structural features;

[0015] The content of the second short text is determined based on the second basic features of the document paragraph and the structural features of the target document;

[0016] The second short text content is input into a preset deep learning model to output the tags of the document paragraphs;

[0017] Based on the tags and the Word document to be indexed, determine the automatically indexed Word document.

[0018] In some embodiments, the step of training a pre-trained language model using the target sample data to obtain a preset deep learning model includes:

[0019] Load the pre-trained language model;

[0020] The target sample data is divided into a training set, a validation set, and a test set.

[0021] The pre-trained language model is trained multiple times using the training set to determine the pre-trained language model after each training session.

[0022] Determine the accuracy of the pre-trained language model on the validation set after each training session;

[0023] Determine the total accuracy of the pre-trained language model on the test set corresponding to the highest accuracy;

[0024] If the accuracy rate exceeds the preset accuracy rate, then the pre-trained language model corresponding to the highest accuracy rate is determined and saved as the preset deep learning model.

[0025] In some embodiments, the test set includes sample input data and real labels; the step of determining the all-correction rate of the pre-trained language model on the test set corresponding to the highest accuracy includes:

[0026] The sample input data in the test set is input into the pre-trained language model that corresponds to the highest accuracy, so as to output the output data corresponding to the sample input data.

[0027] Collect samples in the test set whose output data and true labels are all identical to the input data of the samples.

[0028] Calculate the proportion of sample documents in the test set whose output data and true labels are completely identical to the sample input data, to the total number of sample documents in the test set, and determine the proportion as the full accuracy rate of the pre-trained language model on the test set corresponding to the highest accuracy.

[0029] In some embodiments, the step of determining the automatically indexed Word document based on the tag and the Word document to be indexed includes:

[0030] From the tags, select tags that meet the preset rules;

[0031] The tags that conform to the preset rules are split to obtain the split tags;

[0032] Identify paragraphs in the Word document to be indexed that do not have preset characteristics and their corresponding underlying tags;

[0033] The document paragraphs are labeled with both unsplit and split tags to obtain the labeled document paragraphs;

[0034] The underlying tags are used to label paragraphs that do not have the preset features, resulting in labeled paragraphs that do not have the preset features.

[0035] The tagged document paragraphs and the tagged paragraphs without preset features are combined to obtain an automatically indexed Word document.

[0036] In some embodiments, the step of extracting the normalized first basic feature from the sample paragraphs in the sample document includes:

[0037] Extract basic features of the sample document from the sample paragraphs in the sample document;

[0038] The basic features of the sample document are normalized to obtain the first normalized basic features.

[0039] In some embodiments, the step of processing the normalized first basic feature to generate semantic sample document structure features includes:

[0040] Based on the text order, paragraph distribution characteristics, and paragraph feature differences in the sample documents, statistical analysis is performed on the first basic feature after normalization to extract the structural features of the sample articles.

[0041] The structural features of the sample articles are semantically described to obtain semantically processed document structural features.

[0042] In some embodiments, prior to the step of obtaining a large number of sample documents, the method further includes:

[0043] Obtain sample data to be processed; wherein, the sample data to be processed includes a large number of documents to be processed;

[0044] The paragraphs to be selected in the document to be processed are marked to obtain the paragraph tags of the paragraphs to be selected;

[0045] Based on the paragraph tags, select documents that meet the preset requirements from the documents to be processed in the sample data to be processed as sample documents.

[0046] Secondly, embodiments of this application also provide an automatic indexing device for Word documents based on a short text classification task, comprising:

[0047] The sample document acquisition unit is used to acquire a large number of sample documents, wherein the sample paragraphs of the sample documents are marked with sample tags;

[0048] The first basic feature extraction unit is used to extract the first basic features after normalization from the sample paragraphs in the sample document.

[0049] The sample document structure feature determination unit is used to process the first basic feature that has been normalized to generate semantic sample document structure features.

[0050] The first short text content determination unit is used to determine the content of the first short text based on the first basic features of the sample paragraph and the structural features of the sample document.

[0051] The target sample data determination unit is used to determine that the first short text content is sample input data and the sample label is a real label, and to use the sample input data and the real label to form target sample data;

[0052] A preset deep learning model determination unit is used to train a pre-trained language model using the target sample data to obtain a preset deep learning model.

[0053] The unit for retrieving the Word document to be indexed is used to retrieve the Word document to be indexed.

[0054] The second basic feature extraction unit is used to extract the basic features of the normalized Word document from the document paragraphs in the Word document to be indexed, wherein the document paragraphs have preset features;

[0055] The target document structure feature determination unit is used to process the basic features of the normalized Word document to generate semantic target document structure features.

[0056] The second short text content determination unit is used to determine the second short text content based on the second basic features of the document paragraph and the structural features of the target document;

[0057] The input unit is used to input the second short text content into a preset deep learning model to output the tags of the document paragraph;

[0058] The automatically indexed Word document determination unit is used to determine the automatically indexed Word document based on the tag and the Word document to be indexed.

[0059] Thirdly, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the automatic indexing method for Word documents based on short text classification tasks.

[0060] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the automatic indexing method for Word documents based on short text classification tasks.

[0061] The above embodiments provide an automatic indexing method for Word documents based on short text classification tasks. This method transforms the automatic indexing task of Word documents into a short text classification task in the field of natural language processing. This task uses a pre-trained language model oriented towards semantic understanding, which can automatically mine the potential features of the data and make full use of the semantic features of the articles in the sample documents. Through the structural features of the sample documents and the first short text content generation algorithm, the structural features of the articles in the sample documents are transformed into semantic features. This allows the pre-trained language model to learn the semantic features of the sample paragraphs while also taking into account the structural features of the articles. The trained preset deep learning model automatically establishes strong dependencies between paragraphs and tags to achieve better indexing results. The method includes: acquiring a large number of sample documents, wherein sample paragraphs of the sample documents are labeled with sample tags; extracting a first basic feature after normalization from the sample paragraphs in the sample documents; processing the first basic feature after normalization to generate semantic sample document structure features; determining a first short text content based on the first basic feature of the sample paragraphs and the sample document structure features; determining the first short text content as sample input data and the sample tags as real tags, and using the sample input data and real tags to form target sample data; training a pre-trained language model using the target sample data to obtain a preset deep learning model; acquiring a Word document to be indexed; extracting basic features of the Word document after normalization from the document paragraphs in the Word document to be indexed, wherein the document paragraphs have preset features; processing the basic features of the Word document after normalization to generate semantic target document structure features; determining a second short text content based on the second basic feature of the document paragraphs and the target document structure features; inputting the second short text content into the preset deep learning model to output the tags of the document paragraphs; and determining the automatically indexed Word document based on the tags and the Word document to be indexed. d document. Attached Figure Description

[0062] Figure 1 An exemplary flowchart illustrates a method for automatic indexing of Word documents based on a short text classification task, according to some embodiments.

[0063] Figure 2 An exemplary flowchart is shown of a training process for a preset deep learning model provided according to some embodiments;

[0064] Figure 3 An exemplary method for prediction using a preset deep learning model is shown according to some embodiments;

[0065] Figure 4An exemplary flowchart is shown, illustrating a method for training a pre-trained language model using the target sample data to obtain a preset deep learning model, according to some embodiments. Detailed Implementation

[0066] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0067] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0068] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0069] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0070] Automatic indexing of Word documents involves tagging paragraphs within the document. This data processing facilitates information storage and retrieval, and forms the foundation for data resource integration and in-depth knowledge mining. For example, if the content of a paragraph contains keywords, that paragraph will be tagged, and the tags can specifically be keywords.

[0071] Currently, most traditional automatic indexing algorithms are rule-based, requiring the development of different rule-based indexing programs for documents with different styles and structures, resulting in low development efficiency. In addition, rule conflicts may occur for documents with relatively complex structures, limiting the scope of application. Furthermore, traditional rule-based automatic indexing algorithms face significant optimization bottlenecks, making it difficult to improve their performance.

[0072] To address the aforementioned technical issues, this application provides an automatic indexing method for Word documents based on a short text classification task. This method transforms the automatic indexing task of Word documents into a short text classification task within the field of natural language processing. This task utilizes a pre-trained language model oriented towards semantic understanding, which can automatically mine the latent features of the data and fully leverage the semantic features of the articles in the sample documents. Through the structural features of the sample documents and a first short text content generation algorithm, the structural features of the articles in the sample documents are transformed into semantic features. This allows the pre-trained language model to learn the semantic features of sample paragraphs while simultaneously considering the structural features of the articles. The trained, pre-defined deep learning model automatically establishes strong dependencies between paragraphs and tags, achieving better indexing results.

[0073] Figure 1 An exemplary flowchart illustrates a method for automatic indexing of Word documents based on a short text classification task, according to some embodiments. Figure 1 This includes the training process of a pre-set deep learning model and the prediction process using that model. After acquiring sample documents, basic features of a Word document are extracted. Then, document structure features are extracted, and short text content is generated. The short text content is then used to train a pre-trained language model to obtain the pre-set deep learning model. When using the pre-set deep learning model, after acquiring the Word document to be indexed, basic features of a Word document are extracted. Then, document structure features are extracted, and short text content is generated. The short text content is then input into the pre-set deep learning model, and finally, labels are output. These labels can be further processed to obtain the indexed Word document.

[0074] The above process will be explained in detail below. Figure 2 An exemplary flowchart of a training process for a preset deep learning model provided according to some embodiments is shown, including steps S100-S600. Figure 3 An exemplary method for predicting a preset deep learning model according to some embodiments is shown, including steps S700-S1200.

[0075] S100. Obtain a large number of sample documents, wherein the sample paragraphs of the sample documents are marked with sample tags.

[0076] In this embodiment of the application, the sample document is a Word document, and the sample tags are obtained by manually annotating the sample paragraphs.

[0077] In some embodiments, to better train the pre-trained language model subsequently, the sample documents are representative and feature-rich data. Specifically, before the step of obtaining a large number of sample documents, the following steps are also included:

[0078] Obtain sample data to be processed; wherein, the sample data to be processed includes a large number of documents to be processed.

[0079] In this embodiment of the application, the document to be processed is a Word document.

[0080] The paragraphs to be selected in the document to be processed are labeled to obtain paragraph tags for the paragraphs to be selected. In some embodiments, the paragraphs to be selected in the document to be processed are all paragraphs in the document to be processed.

[0081] In some embodiments, the selected paragraphs in the document to be processed can be labeled manually to ensure the accuracy of the paragraph labels.

[0082] Based on the paragraph tags, select documents that meet the preset requirements from the documents to be processed in the sample data to be processed as sample documents.

[0083] Since a highly accurate deep learning model can only be obtained by training a pre-trained language model using relatively reasonable and comprehensive samples, it is necessary to select sample documents with comprehensive article structure features and paragraph semantic features based on paragraph tags, i.e., documents that meet the preset requirements. In some embodiments, the step of selecting documents that meet the preset requirements as sample documents includes first excluding documents where any one of the following is equal to 0: title number, abstract number, keyword number, main text paragraph number, or reference paragraph number. These numbers can be determined based on paragraph tags. Then, from the remaining documents, retain those containing metadata and main text, and whose total number of paragraphs is greater than a preset number (30). These retained documents are used as sample documents, thus selecting sample documents with relatively comprehensive article structure features and paragraph semantic features. The metadata refers to data in the document other than the main text. The selected documents are then used as the training set.

[0084] S200. Extract the first basic feature from the sample paragraphs in the sample document.

[0085] In this embodiment, blank lines, images, tables, and formulas in Word documents are directly extractable internal attributes and do not require analysis using the automatic indexing method described in this embodiment. However, considering that these features may have some correlation with other paragraphs, they are also extracted as paragraphs to be selected. The sample tags in the entire sample document are sorted according to the actual text order to ensure the structure and sequence of the article itself.

[0086] In this embodiment of the application, the first basic feature includes article number, paragraph number, sample tag, font, font size, alignment, line spacing, outline level, whether it is bold, whether it is italic, word count, whether it is body text font, whether it is body text font size, language, and text content. The article number refers to the number of the sample document in the target sample data.

[0087] When extracting the first basic feature from sample paragraphs in the sample document, the extraction is performed on a whole-document basis according to the OOXML standard and OLE2 format specification. The first basic features of the sample paragraphs are then sorted according to the textual order of the sample document.

[0088] In some embodiments, the step of extracting the first basic feature from the sample paragraphs in the sample document includes:

[0089] Extract basic features of the sample document from the sample paragraphs in the sample document;

[0090] The basic features of the sample document are normalized to obtain the first basic feature.

[0091] In this embodiment of the application, since the sample document is a Word document and the content in a Word document is diverse, it is necessary to standardize the basic features of the sample document extracted from the sample paragraphs to obtain the first basic features, so that they can be used to train the pre-trained language model to obtain the preset deep learning model that accurately determines the labels.

[0092] Specifically, the basic features of the sample document include fonts. Font standardization involves classifying hundreds of fonts and clustering them into a smaller number of commonly used fonts based on font characteristics and usage habits. When a font that is not among the commonly used fonts exists in the basic features of a Word document, it is modified to a commonly used font.

[0093] The basic features of the sample document include font size. The standardization of font size means that there are two font size forms: Chinese characters and numbers. In this embodiment of the application, according to the standard font size correspondence table, both forms are converted into numerical font sizes.

[0094] The basic features of the sample document include text content. The normalization of text content means that the length of the text content of the sample paragraph is limited, and only a preset number of characters of the text content are taken. For example, the first 128 non-empty characters of the text content are taken.

[0095] In addition to the above-mentioned standardization of font, font size and text content, other standardization processes can also be included, specifically including the standardization of alignment, line spacing, outline level, whether to bold, whether to italicize, language and other content.

[0096] In this embodiment of the application, the basic features of the processed sample document are used as the first basic feature.

[0097] S300. Determine the sample document structure features of the sample paragraph.

[0098] In this embodiment of the application, the sample document structural features refer to the association features between paragraphs established based on the article structure and paragraph association in the sample document. For example, the sample document structural features include the preceding text features, the following text features, the location, whether it is body text font, whether it is body text font size, the difference between body text font size and body text font size, paragraph length and the difference between the number of words above and below.

[0099] Here, the context feature refers to the label of the previous sample paragraph in the sample document. Using the label of the previous sample paragraph as the context feature aims to construct the relationship between the current sample paragraph and the context of the article in the sample document.

[0100] The context features refer to taking known blank lines, tables, formulas, and graphs as context features of the current sample paragraph, with the aim of constructing a partial relationship between the current sample paragraph and the context of the article in the sample document.

[0101] In practical applications, the document paragraphs in the Word document to be indexed are sequentially tagged using a pre-defined deep learning model, as detailed below. Since the tags for blank lines, tables, formulas, and figures can be known in advance from the underlying data of the Word document to be indexed, but the tags for paragraphs that are not blank lines, tables, formulas, or figures cannot be known in advance, if the following text of the current document paragraph is a blank line, table, formula, or figure, then the target document structure features of the current document paragraph contain the following text feature. If the following text of the current document paragraph is not a blank line, table, formula, or figure, then the target document structure features do not include the following text feature when obtaining the target document structure features corresponding to the current document paragraph. Therefore, when determining the samples for training the model, similarly, if the following text of the current sample paragraph is a blank line, table, formula, or figure, then the sample document structure features of the current sample document contain the following text feature. If the following text of the current sample paragraph is not a blank line, table, formula, or figure, then the sample document structure features do not include the following text feature when obtaining the sample document structure features corresponding to the current sample paragraph.

[0102] The location refers to the position of the current sample paragraph within the sample document, such as before, in the middle, or after it. This is intended to establish the relationship between the current sample paragraph and the overall text in the sample document.

[0103] For example, when the current sample paragraph is in the first 20% of all sample paragraphs in the sample document, it is considered to be in the first position of the sample document; when the current sample paragraph is in the last 20% of all sample paragraphs in the sample document, it is considered to be in the last position of the sample document; when the current sample paragraph is between the first 20% and the first 80% of all sample paragraphs in the sample document, it is considered to be in the middle position of the sample document.

[0104] Whether it is the body text font and whether it is the body text size refers to the fact that the sample document includes metadata and body text data. The metadata is other data in the sample document besides the body text data. The font size of the current sample paragraph is compared with the font size of the body text to determine the difference between the current sample paragraph and the body text.

[0105] The difference between the font size of the current sample paragraph and the font size of the main text refers to the difference between the font size of the current sample paragraph and the font size of the main text, which is intended to obtain the difference relationship between the font size features of the current sample paragraph and the main text.

[0106] Paragraph length refers to the length of the current sample paragraph, which is divided into three categories: long, medium, and short, to roughly construct the characteristics of the length of the current sample paragraph.

[0107] The word count difference refers to the difference in word count between the current sample paragraph and the previous sample paragraph, which aims to obtain the correlation features between the two sample paragraphs from the perspective of word count.

[0108] S400. Determine the content of the first short text based on the first basic features of the sample paragraph and the structural features of the sample document.

[0109] The first short text content is composed of some or all of the processed first basic features and some or all of the sample document structure features.

[0110] For example, the first short text content can be short text content concatenated in the form of commas, according to the preceding text features, location, paragraph length, difference between the font size and the body text, difference between the number of words and the preceding text, language, font, font size, alignment, line spacing, outline level, whether it is bold, whether it is italic, following text features, whether it is the body text font, whether it is the body text font, and the order of the text content.

[0111] In this embodiment, to reduce the burden of model training and facilitate the convergence of the preset deep learning model parameters, it is necessary to normalize the basic features of the sample document to obtain the first basic features. Based on the first basic features, further feature processing is performed to generate semantic sample document structural features. The first basic features and the sample document structural features are then merged into the first short text content, achieving the normalization of the basic features and the semanticization of the structural features of the article in the sample document.

[0112] S500: Determine that the first short text content is sample input data and the sample label is a real label, and use the sample input data and the real label to form target sample data.

[0113] In this embodiment of the application, the first short text content corresponding to the sample paragraph in each sample document is used as the sample input data, and the sample label corresponding to the sample paragraph is used as the real label, so that a large amount of target sample data can be obtained.

[0114] S600. The pre-trained language model is trained using the target sample data to obtain a preset deep learning model.

[0115] In some embodiments, Figure 4 An exemplary flowchart illustrates a method for training a pre-trained language model using the target sample data to obtain a preset deep learning model, according to some embodiments. The steps of training the pre-trained language model using the target sample data to obtain the preset deep learning model include S601-S606.

[0116] S601, Load the pre-trained language model.

[0117] The deep learning framework used in this embodiment is PaddlePaddle, employing a pre-trained language model oriented towards semantic understanding. Here, ernie-3.0-base-zh is selected as the main model. ERNIE 3.0 is a knowledge-enhanced multi-paradigm unified pre-trained model that integrates autoregressive and autoencoder networks and introduces large-scale knowledge graph-type data, resulting in excellent performance in understanding tasks. This model is applied to the processed target sample data for model training, adaptively learning the article structure and the correspondence between each real label, completing iterative training, obtaining the preset deep learning model, and finally saving the model parameters of the preset deep learning model.

[0118] In some embodiments, a pre-trained language model is loaded into a memory structure.

[0119] S602. Divide the target sample data to obtain a training set, a validation set, and a test set.

[0120] The target sample data is divided according to a preset ratio, for example, an 8:1:1 ratio, to obtain a training set, a validation set, and a test set. Then, hyperparameters such as the batch size of the pre-trained language model, the maximum truncation length of the text sequence, and the text padding value are set, and the dataset is packaged.

[0121] S603. The pre-trained language model is trained multiple times using the training set to determine the pre-trained language model after each training session.

[0122] In this embodiment, a fine-tuning optimization strategy is set, and evaluation metrics are integrated. During the training of the pre-trained language model, a loss function is needed to quantify the difference between the predicted and true values; the model employs the cross-entropy loss function. An optimization algorithm is needed to iterate the model parameters a finite number of times to obtain a better model, thereby minimizing the risk function value; here, the AdamW optimizer is selected.

[0123] S604. Determine the accuracy of the pre-trained language model on the validation set after each training session.

[0124] Accuracy is used as the evaluation metric on the validation set. Accuracy is a metric used to evaluate classification models.

[0125] S605. Determine the perfect accuracy rate of the pre-trained language model on the test set corresponding to the highest accuracy rate.

[0126] In this embodiment of the application, a validation set is used to select the pre-trained language model with the highest accuracy from the pre-trained language models obtained from multiple training sessions.

[0127] In some embodiments, the test set includes sample input data and real labels; the step of determining the all-correction rate of the pre-trained language model on the test set corresponding to the highest accuracy includes:

[0128] The sample input data in the test set is input into the pre-trained language model that corresponds to the highest accuracy, so as to output the output data corresponding to the sample input data.

[0129] Collect samples in the test set whose output data and true labels are all identical to the input data of the samples.

[0130] Calculate the proportion of sample documents in the test set whose output data and true labels are completely identical to the sample input data, to the total number of sample documents in the test set, and determine the proportion as the full accuracy rate of the pre-trained language model on the test set corresponding to the highest accuracy.

[0131] In this embodiment of the application, the Precision rate is used as the evaluation metric on the test set, which is the ratio of the number of paragraphs in the entire article that are completely correctly indexed (i.e., the number of sample documents in the sample document whose true tags are the same as the output data corresponding to the sample input data) to the total number of articles (i.e. the total number of sample documents in the test set).

[0132] S606. If the accuracy rate exceeds the preset accuracy rate, then determine and save the pre-trained language model corresponding to the highest accuracy rate as the preset deep learning model.

[0133] In this embodiment of the application, when the accuracy rate exceeds the preset accuracy rate, it is determined that the pre-trained language model corresponding to the current highest accuracy rate meets the requirements and can be used as the preset deep learning model in actual application.

[0134] In this embodiment of the application, learning a pre-trained language model to obtain a preset deep learning model means that, based on the processed target sample data, a pre-trained language model oriented towards semantic understanding is applied to allow the model to adaptively learn the correspondence between the article content and each sample label, complete iterative training, and finally save the model parameters of the preset deep learning model.

[0135] In this embodiment, hyperparameters such as the maximum learning rate, number of training epochs, learning rate warm-up ratio, and weight decay coefficient are set during the training process of the pre-trained language model, and then training is started. During training, after each training epoch, the accuracy of the model on the validation set is determined, the training result of the model with the best evaluation result is selected, and the full accuracy of the model with the best result on the test set is determined. If it exceeds the preset full accuracy, the optimal model training result is used as the preset deep learning model.

[0136] S700, Obtain the Word document to be indexed.

[0137] In this embodiment of the application, the Word document to be indexed is a Word document of type when the preset deep learning model in this embodiment of the application is actually applied. The preset deep learning model is used to index the Word document to be indexed to obtain the indexed Word document.

[0138] S800. Extract a second basic feature from the document paragraphs in the Word document to be indexed; wherein the document paragraphs have preset features.

[0139] In this embodiment of the application, the Word document to be indexed may include document paragraphs, blank lines, images, tables, and formulas. The document paragraphs refer to paragraphs in the Word document to be indexed that are not blank lines, images, tables, or formulas.

[0140] Since blank lines, tables, formulas, and graphs can be determined based on the Word document to be indexed from the sample document, when indexing the Word document to be indexed using a preset deep learning model, the paragraphs containing blank lines, images, tables, and formulas do not need to be indexed.

[0141] In this embodiment of the application, the second basic features are extracted sequentially from all document paragraphs in the Word document to be indexed.

[0142] The second basic feature includes paragraph numbering, font, font size, alignment, line spacing, outline level, whether it is bold, whether it is italic, word count, whether it is body text font, whether it is body text font size, language, and text content.

[0143] In some embodiments, the difference between the second basic feature and the first basic feature is that the second basic feature does not include the article number.

[0144] In some embodiments, the step of extracting the second basic feature from the document paragraphs in the Word document to be indexed includes:

[0145] Extract the basic features of the Word document from the document paragraphs in the Word document to be indexed;

[0146] The basic features of the Word document are standardized to obtain the second basic features.

[0147] In this embodiment of the application, the step of extracting the second basic feature from the document paragraphs in the Word document to be indexed is similar to the step of extracting the first basic feature from the sample paragraphs in the sample document described above, and will not be repeated here.

[0148] S900. Determine the target document structure features corresponding to the document paragraph.

[0149] In this embodiment of the application, the target document structural features refer to the association features between paragraphs established based on the article structure and paragraph relationships in the Word document to be indexed.

[0150] In this embodiment of the application, the structural features of the target document include the preceding text features, the following text features, the location, whether it is body text font, whether it is body text font size, the difference between body text font size and body text font size, paragraph length and the difference between the number of characters above and below.

[0151] In this embodiment of the application, the process of determining the target document structure features corresponding to the document paragraph is similar to the process of determining the sample document structure features of the sample paragraph described above, and will not be repeated here.

[0152] S1000. Determine the content of the second short text based on the second basic features of the document paragraph and the structural features of the target document.

[0153] In this embodiment of the application, some or all of the second basic features and some or all of the target document structure features are selected and combined to generate the second short text content.

[0154] For example, the second short text content can be short text content spliced ​​together in the form of commas, according to the preceding text features, location, paragraph length, difference between the font size and the body text, difference between the number of words and the preceding text, language, font, font size, alignment, line spacing, outline level, whether it is bold, whether it is italic, following text features, whether it is the body text font, whether it is the body text font, and the order of the text content.

[0155] S1100. Input the content of the second short text into a preset deep learning model to output the tags of the document paragraphs.

[0156] In this embodiment, the second short text content corresponding to all document paragraphs in the Word document to be indexed is sequentially put into a preset deep learning model to output the label of each document paragraph.

[0157] It's important to note that when determining the labels of document paragraphs, the labels of the previous document paragraph can also be used as contextual features in the second short text content of the current document paragraph. Specifically, the second short text content corresponding to each document paragraph is sequentially fed into the model for inference, and the inference results are used as contextual features for the next paragraph for prediction, until the end of the document.

[0158] In some embodiments, a preset deep learning model may be loaded before extracting the second basic features from the document paragraphs in the Word document to be indexed.

[0159] S1200. Based on the label and the Word document to be indexed, determine the automatically indexed Word document.

[0160] In this embodiment, the tags are indexed into the Word document in the form of bookmarks or annotations, resulting in an automatically indexed Word document.

[0161] In some embodiments, the step of determining the automatically indexed Word document based on the tag and the Word document to be indexed includes:

[0162] From the tags, select tags that meet the preset rules;

[0163] The tags that conform to the preset rules are split to obtain the split tags;

[0164] In some embodiments, the first tag determined by a document paragraph may contain multiple tags combined together. In one example, the metadata might label the article code, classification number, and document identification code as a single tag, which can be split into separate tags: article code, classification number, and document identification code. In another example, the Chinese title and Chinese subtitle might be labeled as a single tag, which can be split into separate tags: one for the Chinese title and the other for the Chinese subtitle. Similarly, in yet another example, the English title and English subtitle might be labeled as a single tag, which can be split into separate tags: one for the English title and the other for the English subtitle.

[0165] Identify paragraphs in the Word document to be indexed that do not have preset characteristics and their corresponding underlying tags.

[0166] In this embodiment, the paragraph without preset features can refer to blank lines, images, tables, and formulas. For example, the underlying tags can be images, tables, and formulas. In some embodiments, the underlying tags do not include blank lines.

[0167] The document paragraphs are labeled with both unsplit tags and split tags to obtain the labeled document paragraphs.

[0168] The underlying tags are used to label paragraphs that do not have preset features, resulting in labeled paragraphs that do not have preset features.

[0169] The tagged document paragraphs and the tagged paragraphs without preset features are combined to obtain an automatically indexed Word document.

[0170] In this embodiment of the application, the annotated document paragraphs and the annotated paragraphs without preset features are combined to form a complete Word document, and the complete Word document is an automatically indexed Word document.

[0171] In some embodiments, the automatically indexed Word document can also be validated to identify errors and send it to staff for manual modification.

[0172] The above embodiments provide an automatic indexing method for Word documents based on short text classification tasks. This method transforms the automatic indexing task of Word documents into a short text classification task within the field of natural language processing. This task utilizes a pre-trained language model oriented towards semantic understanding, which can automatically mine the latent features of the data and fully leverage the semantic features of the articles in the sample documents. Through the structural features of the sample documents and a first short text content generation algorithm, the structural features of the articles in the sample documents are transformed into semantic features. This allows the pre-trained language model to learn the semantic features of the sample paragraphs while also considering the structural features of the articles. The trained, pre-defined deep learning model automatically establishes strong dependencies between paragraphs and tags to achieve better indexing results. The method includes: acquiring a large number of sample documents, wherein sample paragraphs of the sample documents are labeled with sample tags; extracting a first basic feature after normalization from the sample paragraphs in the sample documents; processing the first basic feature after normalization to generate semantic sample document structure features; determining a first short text content based on the first basic feature of the sample paragraphs and the sample document structure features; determining the first short text content as sample input data and the sample tags as real tags, and using the sample input data and real tags to form target sample data; training a pre-trained language model using the target sample data to obtain a preset deep learning model; acquiring a Word document to be indexed; extracting basic features of the Word document after normalization from the document paragraphs in the Word document to be indexed, wherein the document paragraphs have preset features; processing the basic features of the Word document after normalization to generate semantic target document structure features; determining a second short text content based on the second basic feature of the document paragraphs and the target document structure features; inputting the second short text content into the preset deep learning model to output the tags of the document paragraphs; and determining an automatically indexed Word document based on the tags and the Word document to be indexed.

[0173] Furthermore, as Figures 1-4 In a specific implementation of the method, this application provides an automatic indexing device for Word documents based on a short text classification task, comprising:

[0174] The sample document acquisition unit is used to acquire a large number of sample documents, wherein the sample paragraphs of the sample documents are marked with sample tags;

[0175] The first basic feature extraction unit is used to extract the first basic features after normalization from the sample paragraphs in the sample document.

[0176] The sample document structure feature determination unit is used to process the first basic feature that has been normalized to generate semantic sample document structure features.

[0177] The first short text content determination unit is used to determine the content of the first short text based on the first basic features of the sample paragraph and the structural features of the sample document.

[0178] The target sample data determination unit is used to determine that the first short text content is sample input data and the sample label is a real label, and to use the sample input data and the real label to form target sample data;

[0179] A preset deep learning model determination unit is used to train a pre-trained language model using the target sample data to obtain a preset deep learning model.

[0180] The unit for retrieving the Word document to be indexed is used to retrieve the Word document to be indexed.

[0181] The second basic feature extraction unit is used to extract the basic features of the normalized Word document from the document paragraphs in the Word document to be indexed, wherein the document paragraphs have preset features;

[0182] The target document structure feature determination unit is used to process the basic features of the normalized Word document to generate semantic target document structure features.

[0183] The second short text content determination unit is used to determine the second short text content based on the second basic features of the document paragraph and the structural features of the target document;

[0184] The input unit is used to input the second short text content into a preset deep learning model to output the tags of the document paragraph;

[0185] The automatically indexed Word document determination unit is used to determine the automatically indexed Word document based on the tag and the Word document to be indexed.

[0186] It should be noted that other corresponding descriptions of the functional units involved in the automatic indexing device for Word documents based on short text classification tasks provided in this embodiment can be found in [reference needed]. Figures 1-4 The corresponding descriptions in [the document] will not be repeated here.

[0187] Based on the above, Figures 1-4 Accordingly, this application embodiment also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method. Figures 1-4The method shown is an automatic indexing method for Word documents based on the short text classification task.

[0188] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause an electronic device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0189] Based on the above, Figures 1-4 To achieve the above objectives, in addition to the methods and virtual device embodiments shown, this application also provides a physical device for automatic indexing of Word documents based on short text classification tasks. Specifically, this physical device can be a computer, smartphone, tablet, smartwatch, server, or network device, etc. The physical device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described... Figures 1-4 The method shown is an automatic indexing method for Word documents based on the short text classification task.

[0190] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0191] Those skilled in the art will understand that the electronic device structure provided in this embodiment does not constitute a limitation on the electronic device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0192] The storage medium may also include an operating device and a network communication module. The operating device is a program that manages and stores the hardware and software resources of the electronic device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between various controls within the storage medium, as well as communication with other hardware and software within the physical device.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0194] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the units or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the units in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be located in one or more apparatuses different from this embodiment, with corresponding changes. The units of the above-described embodiment can be combined into one unit, or further divided into multiple sub-units.

[0195] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for automatic indexing of Word documents based on short text classification tasks, characterized in that, include: Obtain a large number of sample documents, wherein sample paragraphs of the sample documents are marked with sample tags; For the sample paragraphs in the sample document, extract the first basic feature after normalization; The first basic feature, after normalization, is processed to generate semantic sample document structure features; The content of the first short text is determined based on the first basic feature of the sample paragraph and the structural feature of the sample document; The first short text content is determined to be sample input data, and the sample label is a real label. The sample input data and the real label are used to form target sample data. The pre-trained language model is trained using the target sample data to obtain a preset deep learning model; Retrieve the Word document to be indexed; For the document paragraphs in the Word document to be indexed, extract the basic features of the Word document after normalization, wherein the document paragraphs have preset features; The basic features of the normalized Word document are processed to generate semantic target document structural features; The content of the second short text is determined based on the second basic features of the document paragraph and the structural features of the target document; The second short text content is input into a preset deep learning model to output the tags of the document paragraphs; Based on the tags and the Word document to be indexed, determine the automatically indexed Word document.

2. The method according to claim 1, characterized in that, The step of training the pre-trained language model using the target sample data to obtain a preset deep learning model includes: Load the pre-trained language model; The target sample data is divided into a training set, a validation set, and a test set. The pre-trained language model is trained multiple times using the training set to determine the pre-trained language model after each training session. Determine the accuracy of the pre-trained language model on the validation set after each training session; Determine the total accuracy of the pre-trained language model on the test set corresponding to the highest accuracy; If the accuracy rate exceeds the preset accuracy rate, then the pre-trained language model corresponding to the highest accuracy rate is determined and saved as the preset deep learning model.

3. The method according to claim 2, characterized in that, The test set includes sample input data and real labels; The step of determining the all-correct rate of the pre-trained language model on the test set corresponding to the highest accuracy includes: The sample input data in the test set is input into the pre-trained language model that corresponds to the highest accuracy, so as to output the output data corresponding to the sample input data. Collect samples in the test set whose output data and true labels are all identical to the input data of the samples. Calculate the proportion of sample documents in the test set whose output data and true labels are completely identical to the sample input data, to the total number of sample documents in the test set, and determine the proportion as the full accuracy rate of the pre-trained language model on the test set corresponding to the highest accuracy.

4. The method according to claim 1, characterized in that, The step of determining the automatically indexed Word document based on the tag and the Word document to be indexed includes: From the tags, select tags that meet the preset rules; The tags that conform to the preset rules are split to obtain the split tags; Identify paragraphs in the Word document to be indexed that do not have preset characteristics and their corresponding underlying tags; The document paragraphs are labeled with both unsplit and split tags to obtain the labeled document paragraphs; The underlying tags are used to label paragraphs that do not have the preset features, resulting in labeled paragraphs that do not have the preset features. The tagged document paragraphs and the tagged paragraphs without preset features are combined to obtain an automatically indexed Word document.

5. The method according to claim 1, characterized in that, The step of extracting the first basic feature after normalization from the sample paragraphs in the sample document includes: Extract basic features of the sample document from the sample paragraphs in the sample document; The basic features of the sample document are normalized to obtain the first normalized basic features.

6. The method according to claim 1, characterized in that, The step of processing the normalized first basic feature to generate semantic sample document structure features includes: Based on the text order, paragraph distribution characteristics, and paragraph feature differences in the sample documents, statistical analysis is performed on the first basic feature after normalization to extract the structural features of the sample articles. The structural features of the sample articles are semantically described to obtain semantically processed document structural features.

7. The method according to claim 1, characterized in that, Before the step of obtaining a large number of sample documents, the following is also included: Obtain sample data to be processed; wherein, the sample data to be processed includes a large number of documents to be processed; The paragraphs to be selected in the document to be processed are marked to obtain the paragraph tags of the paragraphs to be selected; Based on the paragraph tags, select documents that meet the preset requirements from the documents to be processed in the sample data to be processed as sample documents.

8. An automatic indexing device for Word documents based on short text classification tasks, characterized in that, include: The sample document acquisition unit is used to acquire a large number of sample documents, wherein the sample paragraphs of the sample documents are marked with sample tags; The first basic feature extraction unit is used to extract the first basic features after normalization from the sample paragraphs in the sample document. The sample document structure feature determination unit is used to process the first basic feature that has been normalized to generate semantic sample document structure features. The first short text content determination unit is used to determine the content of the first short text based on the first basic features of the sample paragraph and the structural features of the sample document. The target sample data determination unit is used to determine that the first short text content is sample input data and the sample label is a real label, and to use the sample input data and the real label to form target sample data; A preset deep learning model determination unit is used to train a pre-trained language model using the target sample data to obtain a preset deep learning model. The unit for retrieving the Word document to be indexed is used to retrieve the Word document to be indexed. The second basic feature extraction unit is used to extract the basic features of the normalized Word document from the document paragraphs in the Word document to be indexed, wherein the document paragraphs have preset features; The target document structure feature determination unit is used to process the basic features of the normalized Word document to generate semantic target document structure features. The second short text content determination unit is used to determine the second short text content based on the second basic features of the document paragraph and the structural features of the target document; The input unit is used to input the second short text content into a preset deep learning model to output the tags of the document paragraph; The automatically indexed Word document determination unit is used to determine the automatically indexed Word document based on the tag and the Word document to be indexed.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the step of automatically indexing Word documents based on a short text classification task as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the automatic indexing method for Word documents based on short text classification tasks as described in any one of claims 1 to 7.

Images