Document content structure-based subject term extraction method

Abstract

The invention relates to a subject term extraction method based on a document content structure, in particular to a subject term extraction method based on the document content structure. The method comprises the steps of obtaining a target text fragment, wherein the target text fragment is a text fragment of a subject term to be extracted; the target text fragment belongs to a subject term extraction model, a subject term extraction result is output and obtained, the subject term extraction result is realized in a phrase or phrase form, and the subject term extraction model is an artificial intelligence model; generating an archive label based on the subject term extraction result, wherein the archive label is associated with the target text fragment and the subject term extraction result. In the process of extracting the content of the text fragment, the subject term extraction model is selected to label the text fragment, and the file label corresponding to the text paragraph is generated in combination with an artificial intelligence technology, so that the efficiency of extracting the text content and generating the label is improved.

Description

Technical Field

[0001] The present application relates to the field of applying natural language processing technology in an archive management system, and in particular to applying natural language processing technology in an archive management system. Background Art

[0002] With the acceleration of informatization, the amount of archival data continues to grow, and how to efficiently process and utilize this data has become an important issue.

[0003] Traditional file management methods mainly rely on manual operations.

[0004] However, the file management method in the related art is inefficient and prone to errors. Summary of the invention

[0005] The present application relates to a method for extracting subject words based on document content structure, which can improve the efficiency of file management. The method is applied in a computer device and comprises: Obtain a target text segment, where the target text segment is a text segment of a subject word to be extracted; The target text segment belongs to the subject word extraction model, and the subject word extraction result is output. The subject word extraction result is realized in the form of phrases or word groups. The subject word extraction model is an artificial intelligence model; The archive tag is generated based on the subject word extraction result, and the archive tag is associated with the target text segment and the subject word extraction result.

[0006] In an optional embodiment, the text segment recognition model includes a text field segmentation module, a part-of-speech tagging module, a syntactic analysis module, and a topic word output module that are interconnected; The target text segment belongs to the subject word extraction model, and the subject word extraction results are output, including: Input the target text fragment into the topic extraction model; Perform field segmentation and extraction on the target text segment through a text field segmentation module to obtain at least two sub-texts; The part-of-speech tagging module is used to tag the subtext with parts of speech and sequence; The syntactic analysis module performs syntactic analysis on the sub-text after the part of speech and sequence marking to obtain the analysis result corresponding to the sub-text; Through the keyword output module, the keyword extraction results are obtained based on the analysis results.

[0007] In an optional embodiment, the target text segment is segmented and extracted by a text field segmentation module to obtain at least two sub-texts, including: Obtain a text segmentation database corresponding to the text field segmentation module; Based on the text segmentation database, the target text segment is segmented and extracted by the text field segmentation module to obtain at least two sub-texts.

[0008] In an optional embodiment, the method further includes: Get a sample text field, where the sample text field is marked with a sample subtext; A text segmentation database is generated based on sample text fields. The text segmentation database is a database based on character recognition technology and machine learning.

[0009] In an optional embodiment, the part-of-speech tagging module is used to tag the subtext with a part-of-speech and a sequence, including: Obtain the part-of-speech corpus corresponding to the part-of-speech tagging module; Based on the occurrence frequency of the sub-text and combined with the part-of-speech corpus, the part-of-speech tagging module is used to tag the sub-text with parts of speech or sequence.

[0010] In an optional embodiment, a syntactic analysis module performs syntactic analysis on the subtext after the part of speech and sequence marking to obtain an analysis result corresponding to the subtext, including: Obtaining a text annotation database corresponding to the syntactic analysis module; Based on the text annotation database, the syntactic analysis module performs syntactic analysis on the sub-text after the part of speech or sequence annotation to obtain the analysis result corresponding to the sub-text.

[0011] In an optional embodiment, the method further comprises: Acquire an initial text annotation database, wherein the data format of the initial text annotation database is adapted to the correspondence between the sentence segments and the syntactic analysis; Obtaining a sample sentence segment, where the sample sentence segment is annotated with a sample syntactic analysis result; The initial text annotation database is trained through sample sentences to obtain a text annotation database.

[0012] In an optional embodiment, a keyword extraction result is obtained based on the analysis result through a keyword output module, including: The analysis results corresponding to the sub-texts are quantitatively screened and the results of subject word extraction are output.

[0013] In an optional embodiment, the archive tags are implemented in a list form.

[0014] In an optional embodiment, the method further comprises: Obtain user keyword preference data; Based on user keyword preference data and keyword extraction results, archive tags are generated.

[0015] The beneficial effects of the technical solution provided by this application include at least: In the process of content extraction from text fragments, the keyword extraction model is used to annotate the text fragments, and combined with artificial intelligence technology, file tags corresponding to the text paragraphs are generated, thereby improving the efficiency of text content extraction and label generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a method for extracting keywords based on document content structure provided by an exemplary embodiment of the present application is shown.

[0018] Figure 2 A flowchart of another method for extracting keywords based on document content structure provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0020] Figure 1 A flowchart of a method for extracting subject words based on document content structure provided by an exemplary embodiment of the present application is shown. The method is described by taking the method applied to a computer device as an example. The method includes: Step 101, obtaining a target text segment.

[0021] In the embodiment of the present application, the target text segment is a text segment from which a keyword is to be extracted.

[0022] In the embodiment of the present application, the computer device acquires the target text segment in the application. Optionally, the computer device receives a file start instruction to acquire the editable target text segment in the file; or the computer device receives a file selection instruction to acquire a preset segment in the programmable file. The embodiment of the present application does not limit the specific length and specific content of the target text segment. Step 102, the target text segment belongs to the subject word extraction model, and the subject word extraction result is output.

[0023] In the embodiment of the present application, the subject word extraction model is implemented as an artificial intelligence model. It should be noted that in the embodiment of the present application, the subject word extraction result is implemented in the form of a phrase or a word group.

[0024] In the embodiment of the present application, the subject word extraction result can be implemented in the form of a list, in which the alternative items of the subject word extraction result and the association probability between the subject word and the target text segment are included.

[0025] In some examples, the subject words are phrases that can be directly obtained from the target text segment. In other examples, the subject words cannot be directly selected from the target text segment, and the present application uses the subject words as phrases that can be directly obtained from the target text segment as an example for explanation.

[0026] Step 103: Generate archive tags based on the keyword extraction results.

[0027] Optionally, the archive tag is associated with the target text segment and the subject word extraction result. In actual application, the archive tag can be displayed in a preset area in the application program to visualize the subject word extraction status of the target text segment.

[0028] Optionally, there is a direct correlation between the keyword extraction result and the archive tag.

[0029] Figure 2 A flow chart of another method for extracting subject words based on document content structure provided by an exemplary embodiment of the present application is shown. The method is described by taking the method applied in a computer device as an example. The method includes: Step 201, obtaining a target text segment.

[0030] This process corresponds to the process shown in step 101 and will not be described in detail here.

[0031] Step 202: Input the target text segment into a keyword extraction model.

[0032] In the embodiment of the present application, the keyword extraction model is implemented as an artificial intelligence model. The embodiment of the present application takes the keyword extraction model as a modular model as an example for explanation. According to the function of the keyword extraction model, it includes a text field segmentation module, a part-of-speech tagging module, a syntactic analysis module and a keyword output module that are interconnected.

[0033] Step 203: Perform field segmentation and extraction on the target text segment through a text field segmentation module to obtain at least two sub-texts.

[0034] The text field segmentation module is the word segmentation module, which is the process of dividing the text in the archive data into at least two independent words or phrases. The main idea of ​​statistical word segmentation is to regard each word as composed of the smallest unit of each word. If the number of times the connected words appear in different texts is greater, it proves that the connected words are likely to be a word. Therefore, the frequency of adjacent words can be used to reflect the reliability of the word. The frequency of the combination of adjacent co-occurring words in the corpus is counted. When the combination frequency is higher than the frequency threshold, it is considered that this word combination will constitute a word. In a possible implementation, the Java tool model class library that implements the hidden Markov algorithm is used to segment the input text.

[0035] That is, in this process, a text segmentation database corresponding to the text field segmentation module is obtained; and based on the text segmentation database, field segmentation and extraction are performed on the target text segment through the text field segmentation module to obtain at least two sub-texts.

[0036] The text segmentation database can be a database obtained based on text recognition technology and artificial intelligence technology, and generated based on active learning technology. In the database acquisition stage, sample text fields are obtained, the sample text fields are annotated with sample sub-texts, and a text segmentation database is generated based on the sample text fields. The text segmentation database is a database based on text recognition technology and machine learning.

[0037] Step 204: tag the subtext with parts of speech and sequence using a part-of-speech tagging module.

[0038] After the text segmentation, the computer device performs the part-of-speech tagging process. Tagging the words after word segmentation with part-of-speech helps to understand the meaning and function of the words in the sentence. The words in the corpus are marked with part-of-speech and word frequency. According to the word segmentation results, the frequency of each part-of-speech in the corpus is queried, which part-of-speech appears most frequently for each part-of-speech in the corpus, as the default part-of-speech of the part-of-speech. In addition, the computer device can further identify the criticality of the corresponding sub-text, so the embodiment of the present application includes annotations for the order.

[0039] That is, in an embodiment of the present application, a computer device obtains a part-of-speech corpus corresponding to the part-of-speech tagging module; and based on the frequency of occurrence of the sub-text and in combination with the part-of-speech corpus, the sub-text is tagged with part-of-speech or sequence through the part-of-speech tagging module.

[0040] Step 205: Perform syntactic analysis on the subtext after the part of speech and sequence marking by a syntactic analysis module to obtain an analysis result corresponding to the subtext.

[0041] By analyzing the structure of sentences and understanding the semantic relationship in sentences, it is helpful to extract key information. The main task of syntactic analysis is to identify the syntactic components contained in sentences and the relationship between syntactic components. Generally, the results of syntactic analysis are represented by syntactic trees. The statistical method automatically learns grammatical rules from a large amount of text data through training models, and has better generalization ability. That is, in this process, a text annotation database corresponding to the syntactic analysis module is obtained; based on the text annotation database, the sub-text after the part of speech or sequence annotation is performed by the syntactic analysis module to obtain the analysis result corresponding to the sub-text. In combination with artificial intelligence technology, an initial text annotation database is obtained, and the data format of the initial text annotation database is adapted to the correspondence between the sentence segment and the syntactic analysis; a sample segment is obtained, and the sample segment is annotated with a sample syntactic analysis result; the initial text annotation database is trained by the sample segment to obtain the text annotation database.

[0042] Step 206, obtaining a keyword extraction result based on the analysis result through a keyword output module.

[0043] This process is the process of outputting the extraction results of the topic words and the corresponding probabilities.

[0044] Step 207, obtaining user keyword preference data.

[0045] In the application usage scenario, users usually have a tendency to select keywords, and such a keyword selection process is usually triggered based on the user's selection operation in the application. Optionally, when the user is operating the cataloging system, the computer device can obtain the user's subject word tendency data.

[0046] Step 208: Generate profile tags based on user keyword preference data and keyword extraction results.

[0047] This process is the process of screening based on user usage tendencies and generating keyword extraction results.

[0048] To sum up, the method provided in the embodiment of the present application, in the process of content extraction for text fragments, selects a keyword extraction model to annotate the text fragments, and combines artificial intelligence technology to generate archive tags corresponding to the text paragraphs, thereby improving the efficiency of text content extraction and label generation.

[0049] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for extracting keywords based on document content structure, characterized in that: The method is applied in a computer device, and the method comprises: Obtaining a target text segment, wherein the target text segment is a text segment of a subject word to be extracted; The target text segment belongs to a subject word extraction model, and a subject word extraction result is outputted, wherein the subject word extraction result is implemented in the form of a phrase or a word group, and the subject word extraction model is an artificial intelligence model; An archive tag is generated based on the subject word extraction result, and the archive tag is associated with the target text segment and the subject word extraction result.

2. The method according to claim 1, characterized in that The text segment recognition model includes a text field segmentation module, a part-of-speech tagging module, a syntactic analysis module and a keyword output module that are interconnected; The target text segment is classified into a subject word extraction model, and the subject word extraction result is outputted, including: Inputting the target text segment into the subject word extraction model; Perform field segmentation and extraction on the target text segment by the text field segmentation module to obtain at least two sub-texts; Performing part of speech and sequence tagging on the subtext by the part of speech tagging module; Performing syntactic analysis on the subtext after part of speech and sequence marking by the syntactic analysis module to obtain an analysis result corresponding to the subtext; The keyword extraction result is obtained based on the analysis result through the keyword output module.

3. The method according to claim 2, characterized in that The text field segmentation module is used to perform field segmentation and extraction on the target text segment to obtain at least two sub-texts, including: Acquire a text segmentation database corresponding to the text field segmentation module; Based on the text segmentation database, the text field segmentation module performs field segmentation and extraction on the target text segment to obtain at least two segments of the sub-text.

4. The method according to claim 3, characterized in that The method further comprises: Acquire a sample text field, wherein the sample text field is annotated with a sample subtext; A text segmentation database is generated based on the sample text field, and the text segmentation database is a database based on character recognition technology and machine learning.

5. The method according to claim 2, characterized in that: The part-of-speech and sequence tagging of the subtext by the part-of-speech tagging module includes: Acquire a part-of-speech corpus corresponding to the part-of-speech tagging module; Based on the occurrence frequency of the sub-text and in combination with the part-of-speech corpus, the sub-text is tagged with the part-of-speech tagging module, that is, the part-of-speech sequence.

6. The method according to claim 2, characterized in that The syntactic analysis module performs syntactic analysis on the subtext after the part of speech and sequence marking to obtain an analysis result corresponding to the subtext, including: Acquire a text annotation database corresponding to the syntactic analysis module; Based on the text annotation database, the syntactic analysis module performs syntactic analysis on the subtext after the part of speech or sequence annotation to obtain an analysis result corresponding to the subtext.

7. The method according to claim 6, characterized in that The method further comprises: Acquire an initial text annotation database, wherein the data format of the initial text annotation database is adapted to the correspondence between the sentence segments and the syntactic analysis; Obtaining a sample sentence segment, wherein the sample sentence segment is annotated with a sample syntactic analysis result; The initial text annotation database is trained by using the sample sentences to obtain the text annotation database.

8. The method according to claim 2, characterized in that: The step of obtaining the subject word extraction result based on the analysis result through the subject word output module includes: The analysis results corresponding to the sub-text are quantitatively screened and output to obtain the subject word extraction results.

9. The method according to claim 1, characterized in that: The archive tags are implemented in a list form.

10. The method according to claim 9, characterized in that The method further comprises: Obtain user keyword preference data; The archive tag is generated based on the user keyword tendency data and combined with the keyword extraction result.

Images