Key information extraction method, device, equipment and readable storage medium

Abstract

The embodiment of the present invention discloses a key information extraction method, device, equipment and readable storage medium. The key information extraction method comprises: performing content parsing on a target text to obtain a parsing result; wherein the parsing result comprises: text content; performing regular matching on the text content and a set field to obtain a regular matching result; according to the regular matching result, determining a key information extraction method, and extracting the key information in the extraction method; wherein the extraction method comprises at least two extraction methods. The embodiment of the present invention realizes the customization of different key information extraction methods for different types of target texts according to their characteristics, thereby effectively improving the efficiency of extracting key information from the target text.

Description

Technical Field

[0001] The present invention relates to the field of document processing technology, and in particular to a key information extraction method, device, equipment and readable storage medium. Background Art

[0002] In today's era of information explosion, the amount of information in texts has also increased significantly, mainly reflected in the wide content and large number of words in the texts. Some systematic reports, such as research reports, company annual reports and analysis reports, can easily have hundreds of thousands to millions of words. Accurately and quickly obtaining key information from these articles and grasping the theme of the articles are of great significance for saving readers' energy and reducing information processing time.

[0003] At present, the key information of articles is mostly obtained by manually reading texts and summarizing and recording them. This method has a high accuracy rate and can obtain relatively high-quality key information for complex texts, but the labor cost is extremely high. There is an urgent need for a method that can accurately and quickly automatically extract key information from articles. Summary of the invention

[0004] The embodiments of the present invention provide a key information extraction method, device, equipment and readable storage medium to achieve the purpose of automatically extracting key information from an article accurately and quickly.

[0005] In a first aspect, an embodiment of the present invention provides a key information extraction method, the method comprising:

[0006] Performing content analysis on the target text to obtain analysis results; wherein the analysis results include: text content;

[0007] Perform regular expression matching on the text content and the set field and obtain a regular expression matching result;

[0008] According to the regular matching result, a method for extracting key information is determined, and the key information is extracted in the extraction method; wherein the extraction method includes at least two extraction methods.

[0009] Optionally, determining a method for extracting key information according to the regular matching result includes:

[0010] Establishing an association relationship between the regular matching result and the method for extracting the key information;

[0011] If the regular expression is matched successfully, the first extraction method of the key information is determined;

[0012] If the regular matching is unsuccessful, the key information is extracted in the second extraction method.

[0013] Optionally, the content parsing of the target text to obtain a parsing result includes:

[0014] Convert the target text into a preset format to obtain a format conversion result;

[0015] A file in a target format is selected from the format conversion result as the parsing result.

[0016] Optionally, extracting key information in the extraction manner includes:

[0017] If the extraction method is determined to be the first extraction method, a key information block is determined, and key information is extracted from the key information block.

[0018] Optionally, performing regular expression matching on the text content and a set field and obtaining a regular expression matching result includes:

[0019] Perform regular expression matching on the text title and the directory field;

[0020] The body text located before the directory field is subjected to regular matching with the summary field and the keyword field, and a regular matching result is obtained.

[0021] Optionally, before parsing the target text to obtain the parsing result, the method further includes:

[0022] Receiving input text and checking the format of the input text;

[0023] If the format of the input text meets the set requirements, the input text is divided into target text.

[0024] Optionally, extracting key information in the extraction manner includes:

[0025] If the extraction method is determined to be the second extraction method, inputting the target text into a language analysis model;

[0026] The key information of the target text is extracted through the language analysis model.

[0027] Optionally, extracting key information of the target text through the language analysis model includes:

[0028] Performing word segmentation processing on the target text to obtain a word segmentation result;

[0029] Inputting the word segmentation result into the language analysis model so that the language analysis model can establish a word vector matrix for the word segmentation result;

[0030] According to the word vector matrix, obtaining the contribution of the words in the word segmentation result to the target text;

[0031] Key information is determined according to the contribution of the words, and the key information is extracted.

[0032] Optionally, before inputting the target text into the language analysis model, the method further includes:

[0033] Counting the number of characters in the target text;

[0034] The language analysis model version is determined according to the number of characters in the target text, wherein the language analysis model version includes: a complete version language analysis model and a simplified version language analysis model.

[0035] Optionally, the complete version of the language analysis model is obtained by training with the word segmentation results of the entire target document as a data set; the simplified version of the language analysis model is obtained by training with the word segmentation results of pre-selected paragraphs in the target document as a data set.

[0036] Optionally, determining key information according to the contribution of the word and extracting the key information includes:

[0037] Sorting the words according to their contribution;

[0038] Extracting the words ranked within a set range as key words; and / or,

[0039] Linearly combining the words to obtain a sentence, and calculating the contribution of the sentence according to the contribution of the words;

[0040] The sentences are sorted, and the sentences ranked within a set range are extracted as key sentences.

[0041] In a second aspect, an embodiment of the present invention further provides a key information extraction device, the device comprising:

[0042] The parsing module is used to parse the target text to obtain a parsing result; wherein the parsing result includes: text content;

[0043] A regular matching module, used to perform regular matching on the text content and the set field and obtain a regular matching result;

[0044] The key information extraction module is used to determine the key information extraction method according to the regular matching result, and extract the key information in the extraction method; wherein the extraction method includes at least two extraction methods.

[0045] In a third aspect, an embodiment of the present invention further provides an electronic device, the device comprising:

[0046] one or more processors;

[0047] A memory for storing one or more programs;

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement a key information extraction method as described in any embodiment of the present invention.

[0049] In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a key information extraction method described in any embodiment.

[0050] In the embodiment of the present invention, the target text is parsed to obtain a parsing result, and the text content in the parsing result is regularly matched with a set field to obtain a regular matching result. According to the regular matching result, a method for extracting key information is determined, and the key information is extracted in the extraction method, thereby achieving the characteristics of different types of target texts, customizing different key information extraction methods for them, and effectively improving the efficiency of extracting key information from the target text. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a key information extraction method in Embodiment 1 of the present invention;

[0052] Figure 2 is a flow chart of a key information extraction method in Embodiment 2 of the present invention;

[0053] Figure 3 is a flow chart of a key information extraction method in Embodiment 3 of the present invention;

[0054] Figure 4 It is a structural schematic diagram of a key information extraction device in Embodiment 4 of the present invention;

[0055] Figure 5 It is a schematic diagram of the structure of an electronic device to which the key information extraction method in an embodiment of the present invention is applicable. DETAILED DESCRIPTION

[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0057] Embodiment 1

[0058] Figure 1is a flowchart of a key information extraction method in Embodiment 1 of the present invention. This embodiment is applicable to extracting key information from an article. The method can be executed by a key information extraction device, which can be implemented in software and / or hardware and can be configured in an electronic device. Figure 1 As shown, the method specifically includes:

[0059] S110, performing content analysis on the target text to obtain analysis results; wherein the analysis results include: text content.

[0060] The target text refers to the text to be processed that meets the key information extraction conditions. The key information extraction conditions here can be requirements for the format of the text or whether the address of the text is valid. The text content is the plain text that does not include the text format information. The content parsing of the target text is mainly to separate the plain text content of the target text from the text format, and obtain the plain text and text format files of the target text respectively.

[0061] In this embodiment, optionally, before parsing the target text to obtain the parsing result, the method further includes: receiving input text and checking the format of the input text; if the format of the input text meets the set requirements, dividing the input text into target text.

[0062] Among them, checking the format of the input text mainly checks whether the format of the input text can be parsed into the format of a plain text file. If the format of the input text meets the set requirements, it means that the input text can be parsed into a plain text file, and the electronic device divides the input text of this format into the target text. Optionally, the set format requirement can be docx format, doc format or unencrypted pdf format. It is not limited here and is determined based on actual conditions. If the format of the input text does not meet the set requirements, it means that the input text cannot be parsed into a plain text file, and the input file of this format will not be used as the target file for extracting key information. Optionally, in addition to checking the format of the input file, when the electronic device receives the storage address information of the input file, the address of the input file is checked to determine whether the input file exists.

[0063] S120, performing regular expression matching on the text content and the set field and obtaining a regular expression matching result.

[0064] Among them, the set field is a field that can represent the location of key information. Generally speaking, an applied article that meets the format specifications, such as an academic paper or a research report, usually uses specific fields to identify the key information of the entire article. For example, an academic paper usually uses fields such as keywords and abstracts as identifiers to explain the key information of the article. Optionally, the number of set fields is at least one, and the content of the set field can be text such as "abstract" or "keywords", which is not limited here and is determined based on actual conditions.

[0065] Here, regular matching is a string matching method used to check whether the text contains a set field. The electronic device performs regular matching on the text content in the parsed result and the set field and obtains a regular matching result. The obtained matching result can be that at least one set field is matched, that is, the regular matching is successful; or no set field is matched, that is, the regular matching fails.

[0066] S130. Determine a method for extracting key information according to the regular matching result, and extract the key information using the extraction method; wherein the extraction method includes at least two extraction methods.

[0067] Among them, key information refers to key words or key sentences that can accurately reflect the main content of the target text.

[0068] In this embodiment, optionally, an association relationship is established between the regular matching result and the method for extracting the key information; if the regular matching is successful, the method for extracting the key information is determined to be the first extraction method; if the regular matching is unsuccessful, the method for extracting the key information is determined to be the second extraction method.

[0069] For different regular matching results, the corresponding key information extraction method is selected. When the regular matching result is a regular matching success, it indicates that the target text is an article that conforms to specific writing format specifications. In view of the relatively clear location of key information in such articles, the electronic device selects a relatively simple and fast first key information extraction method to extract key information from the target document. On the contrary, when the regular matching result is a regular matching failure, it indicates that the target text is an article without obvious key information identification. In view of the uncertain location of key information in such articles, the electronic device selects a second key information extraction method to extract key information from the target document. It is worth noting that the first key information extraction method here is different from the second key information extraction method.

[0070] The purpose of regular matching is to query whether there are key information markers in the target text. For texts with obvious key information markers, the key information can be quickly located and a relatively fast and direct extraction method can be selected to extract the key information from the target text. For written texts without obvious key information markers, the entire text needs to be processed to extract the key information.

[0071] In the embodiment of the present invention, the target text is parsed to obtain a parsing result, and the text content in the parsing result is regularly matched with a set field to obtain a regular matching result. According to the regular matching result, a method for extracting key information is determined, and the key information is extracted in the extraction method, thereby achieving the characteristics of different types of target texts, customizing different key information extraction methods for them, and effectively improving the efficiency of extracting key information from the target text.

[0072] Embodiment 2

[0073] Figure 2 This is a flow chart of the key information extraction method in the second embodiment of the present invention. This embodiment is further optimized on the basis of the above embodiment. The specific optimization is: the content parsing of the target text to obtain the parsing result includes: converting the format of the target text into a preset format to obtain a format conversion result; selecting a file in the target format from the format conversion result as the parsing result.

[0074] Optionally, the regular matching of the text content with the set field and obtaining a regular matching result includes: regular matching of the text title with the directory field; regular matching of the main text located before the directory field with the summary field and the keyword field, and obtaining a regular matching result.

[0075] like Figure 2 As shown, the method includes:

[0076] S210: Convert the format of the target text into a preset format to obtain a format conversion result.

[0077] The electronic device converts the format of the target text into a preset format, and the obtained format conversion result includes a plain text corresponding to the target text without typesetting formats such as font type, font size, and space spacing.

[0078] S220: Select a file in a target format from the format conversion result as the parsing result. The parsing result includes: text content.

[0079] Among them, the file in the target format includes the plain text content corresponding to the target document, and the file is the parsing result finally obtained by parsing the target text. In a specific application scenario, when the format of the target text is a text in docx format, the format of the target text is converted to a preset format, that is, the extension of the target text is converted from docx to a file in ZIP format. The file in ZIP format is the format conversion result. Next, the file in ZIP format is decompressed, and it can be seen that there is a folder such as Word in the decompressed folder, which contains most of the content of the Word document. Select XML (extensible in markup language) and then parse XML data to obtain a plain text format text. The reason why the method is preferably adopted is that the way of converting to XML data first can ensure the integrity of the original data, and will not lose part of the original data or data format. In addition, since XML parsing is not limited by the quality of third-party parsing, XML data is converted into plain text format data and can be defined according to project requirements, so that it is not easy to lose text and font type, font size and space spacing and other typesetting format information.

[0080] S230: Perform regular expression matching on the text title and the directory field.

[0081] For a more formal research report, the author will generally set up "Abstract" or "Keywords" fields as key information identifiers in certain paragraphs at the beginning of the article to mark the location of the key information in the article for the convenience of readers. However, it is inevitable that the main body of the article is also likely to include fields such as "Abstract" or "Keywords". The "Abstract" field or "Keywords" field alone cannot accurately locate the location of the key information in the article. In order to obtain a more accurate key information location, optionally, before the text content setting field is regularly matched, the text title is regularly matched with the directory field to define the scope of regular matching of the main text with the abstract field and the keyword field.

[0082] S240: Perform regular expression matching on the body text located before the directory field, the summary field and the keyword field, and obtain a regular expression matching result.

[0083] The table of contents field refers to the table of contents field with the title font. Selecting the text before the table of contents field for regular matching with the "Abstract" and "Keywords" fields can effectively reduce the interference of the "Abstract" and "Keywords" fields that appear repeatedly in the text at other locations on the key information identification process, thereby improving the accuracy of key information extraction.

[0084] If the main text before the directory field contains any field of summary or keyword, a regular match success result is obtained; on the contrary, if the main text before the directory field does not include any field of summary or keyword, a regular match failure result is obtained.

[0085] S250, determining a method for extracting key information according to the regular matching result, and if the extraction method is determined to be the first extraction method, determining a key information block, and extracting key information from the key information block.

[0086] If the extraction method is determined to be the first extraction method, it indicates that the regular match is successful, and the body text before the directory field contains either a summary or a keyword field. At this time, the electronic device determines the key information block according to the location of the summary field or the keyword field. Optionally, the key information block is the paragraph identified by the summary field or the keyword field, and the target text extraction process can be completed by extracting the paragraph from the target text.

[0087] S260: Determine a method for extracting key information according to the regular matching result; if the extraction method is determined to be the second extraction method, input the target text into a language analysis model.

[0088] If the extraction method is determined to be the second extraction method, it indicates that the regular match fails, and the body text before the directory field does not contain any abstract or keyword field, and the target text does not have obvious key information identification. At this time, the key information in the target document can be extracted by natural language processing. Optionally, the target text is input into a language analysis model. Exemplarily, the language analysis model can be a TF-IDF model, a TextRank model, a BERT model, or a Word2Vec word vector model.

[0089] It is worth noting that step S250 and step S260 have no logical order, but are two parallel steps. In order to concisely show the steps of the embodiment of the present invention, they are drawn sequentially in the flowchart corresponding to the embodiment of the present invention.

[0090] S270: Extract key information of the target text through the language analysis model.

[0091] First, the target text is preprocessed. In this embodiment, optionally, the target text is segmented to obtain a segmentation result.

[0092] Because words are the smallest meaningful language components that can act independently. Chinese is written in characters, and there are no obvious distinguishing marks between words. Therefore, Chinese natural language processing usually involves first segmenting the strings in Chinese text into reasonable word sequences, and then performing other analysis and processing on this basis. Segmenting the target text can cut off context coupling and reduce the impact of word order on key information extraction. Optionally, use jieba to perform word-level segmentation on the article. Among them, jieba is currently a Python Chinese word segmentation component.

[0093] In this embodiment, optionally, the word segmentation result is input into the language analysis model so that the language analysis model establishes a word vector matrix for the word segmentation result; based on the word vector matrix, the contribution of the words in the word segmentation result to the target text is obtained; based on the contribution of the words, key information is determined and the key information is extracted.

[0094] Exemplarily, when the language analysis model is the Word2Vec model, the contribution of a word to the target text is to cluster the word vector established by the word segmentation result of the Word2Vec model through the K-Means algorithm, select the cluster center as a main keyword of the article, and calculate the similarity obtained by the distance between each word vector and the cluster center.

[0095] In this embodiment, optionally, the words are sorted according to the contribution of the words; the words ranked within a set range are extracted as key words; and / or the words are linearly combined to obtain sentences, and the contribution of the sentences is calculated according to the contribution of the words; the sentences are sorted, and the sentences ranked within a set range are extracted as key sentences.

[0096] Optionally, according to the contribution of the words, the words are sorted in descending order of contribution, and the words ranked in the top few or dozens of contribution are selected as key words. This is not limited here and is determined based on actual conditions. A sentence is composed of words. After the contribution of the words is determined, the contribution of the sentence can be obtained based on the contribution of the words. Similarly, according to the contribution of the sentences, the sentences can be sorted in descending order of contribution, and the sentences ranked in the top few or dozens of contribution are selected as key sentences.

[0097] The embodiment of the present invention defines the scope of key information identification by firstly performing regular matching of the text title with the directory field, and then performing regular matching of the main text located before the directory field with the summary field and the keyword field when matching the text content with the set field, so that when the regular matching of the text content with the set field is successful within the defined scope, the key information block is determined and then the key information is extracted from the key information block, thereby effectively improving the accuracy of key information extraction. In addition, the embodiment of the present invention also provides a method for extracting key information through a language analysis model, which can be applied to when the regular matching of the text content with the set field fails, and a specific extraction method is selected for different situations to effectively improve the efficiency of key information extraction.

[0098] Embodiment 3

[0099] Figure 3 It is a flow chart of the key information extraction method in the third embodiment of the present invention. This embodiment is further optimized on the basis of the above embodiment. The specific optimization is: before the target text is input into the language analysis model, it also includes: counting the number of characters of the target text; determining the language analysis model version according to the number of characters of the target text, wherein the language analysis model version includes: a complete language analysis model and a simplified language analysis model.

[0100] S310, performing content analysis on the target text to obtain analysis results; wherein the analysis results include: text content;

[0101] S320, performing regular expression matching on the text content and the set field and obtaining a regular expression matching result;

[0102] S330, counting the number of characters in the target text;

[0103] The target file contains non-text information such as tables and images. The existence of such information does not provide effective information when extracting key information based on text, but will affect the electronic device's judgment of the target text size. By counting the number of characters in the target text, the size of the target text can be accurately obtained.

[0104] S340: Determine a language analysis model version according to the number of characters in the target text, wherein the language analysis model version includes: a complete language analysis model and a simplified language analysis model.

[0105] When the target text has a large number of characters, that is, the target text is very long, in this case, inputting the segmentation results of the entire target text into the language analysis model will incur a huge time overhead in extracting key information through the language analysis model. In order to save time costs, the segmentation results of paragraphs with a higher probability of key information appearing are selected and input into the language analysis model, and the contribution of the words in this part is ranked. At this time, the selected language model is a simplified version of the language analysis model; correspondingly, when the target text has a moderate number of characters, the segmentation results of the entire target document are input into the language analysis model, the contribution of the words in the entire document is ranked, and key words are selected from them.

[0106] Among them, determining the language analysis model version according to the number of characters of the target text can determine whether the number of characters of the target text is greater than a set threshold. If it is greater than or equal to the set threshold, it can be inferred that the number of characters of the target text is large; correspondingly, if it is less than the set threshold, it can be inferred that the number of characters of the target text is moderate. Exemplarily, the set threshold can be 90,000, which is not limited here and is determined according to actual conditions.

[0107] In this embodiment, optionally, the complete language analysis model is trained using the word segmentation results of the entire target document as a data set; the simplified language analysis model is trained using the word segmentation results of preselected paragraphs in the target document as a data set.

[0108] Among them, the pre-selected paragraphs are paragraphs with a high probability of key information appearing. For example, considering that the key information of the target text is currently mostly contained in the table of contents, the beginning and the end of the article, the pre-selected paragraphs can be the table of contents, the beginning and the end of the article. The language analysis model is trained using two data sets, so that the trained language analysis model can more accurately determine the word vector of the text when processing the target text within a specific character range, thereby improving the efficiency of key information extraction.

[0109] S350, inputting the target text into a language analysis model;

[0110] S360: extract key information of the target text through the language analysis model.

[0111] The embodiment of the present invention utilizes different data sets to train a language analysis model, forming a complete version and a simplified version of the language analysis model, and determines the language analysis model version to be used when extracting key information based on the number of characters in the target text, thereby greatly saving the number of processed texts, effectively reducing the time cost of training the language analysis model, and improving the efficiency of extracting key information.

[0112] Embodiment 4

[0113] Figure 4is a schematic diagram of the structure of a key information extraction device in Embodiment 6 of the present invention. This embodiment is applicable to extracting key information from an article. The method can be executed by a key information extraction device, which can be implemented in software and / or hardware and can be configured in an electronic device. Figure 4 As shown, the device 400 may include: a parsing module 410 , a regular matching module 420 , and a key information extraction module 430 .

[0114] The parsing module 410 is used to parse the target text to obtain a parsing result; wherein the parsing result includes: text content;

[0115] A regular expression matching module 420 is used to perform regular expression matching on the text content and a set field and obtain a regular expression matching result;

[0116] The key information extraction module 430 is used to determine a key information extraction method according to the regular matching result, and extract the key information in the extraction method; wherein the extraction method includes at least two extraction methods.

[0117] In the embodiment of the present invention, the target text is parsed to obtain a parsing result, and the text content in the parsing result is regularly matched with a set field to obtain a regular matching result. According to the regular matching result, a method for extracting key information is determined, and the key information is extracted in the extraction method, thereby achieving the characteristics of different types of target texts, customizing different key information extraction methods for them, and effectively improving the efficiency of extracting key information from the target text.

[0118] Optionally, the key information extraction module 430 includes: an association relationship establishment submodule, used to establish an association relationship between the regular matching result and the key information extraction method; an extraction method determination first submodule, used to determine the key information extraction method as a first extraction method if the regular matching is successful; and an extraction method determination second submodule, used to determine the key information extraction method as a second extraction method if the regular matching is unsuccessful.

[0119] Optionally, the parsing module 410 includes: a format change submodule, used to transform the format of the target text into a preset format to obtain a format transformation result; and a target format file selection submodule, used to select a file in a target format from the format transformation result as the parsing result.

[0120] Optionally, the key information extraction module 430 includes: a first key information extraction submodule, configured to determine a key information block and extract key information from the key information block if the extraction method is determined to be the first extraction method.

[0121] Optionally, the regular matching module 420 includes: a first regular matching submodule, used to perform regular matching on the text title with the directory field; a second regular matching submodule, used to perform regular matching on the body text located before the directory field with the summary field and the keyword field, and obtain a regular matching result.

[0122] Optionally, the device 400 also includes: a text format checking module, which is used to receive input text and check the format of the input text before parsing the target text to obtain the parsing result; and a target text division module, which is used to divide the input text into target text if the format of the input text meets the set requirements.

[0123] Optionally, the key information extraction module 430 includes: a target text input submodule, used to input the target text into the language analysis model if the extraction method is determined to be the second extraction method; and a second key information extraction submodule, used to extract key information of the target text through the language analysis model.

[0124] Optionally, the second key information extraction submodule includes: a word segmentation unit, used to perform word segmentation on the target text to obtain a word segmentation result; a word segmentation result input unit, used to input the word segmentation result into the language analysis model, so that the language analysis model can establish a word vector matrix for the word segmentation result; a contribution analysis unit, used to obtain the contribution of the words in the word segmentation result to the target text according to the word vector matrix; a key information extraction unit, used to determine key information according to the contribution of the words, and extract the key information.

[0125] Optionally, the device 400 further includes: a character count module, which is used to count the number of characters of the target text before inputting the target text into the language analysis model; a language analysis model version determination module, which is used to determine the language analysis model version according to the number of characters of the target text, wherein the language analysis model version includes: a complete language model and a simplified language model. The complete language analysis model is trained using the word segmentation results of the entire target document as a data set; the simplified language analysis model is trained using the word segmentation results of a preselected paragraph in the target document as a data set.

[0126] Optionally, the key information extraction unit includes: a word sorting subunit, used to sort the words according to the contributions of the words; a key word extraction subunit, used to extract the words ranked within a set range as key words; and / or a sentence contribution calculation subunit, used to linearly combine the words to obtain sentences, and calculate the contribution of the sentences based on the contributions of the words; a key sentence extraction unit, used to sort the sentences and extract the sentences ranked within a set range as key sentences.

[0127] A key information extraction device provided in an embodiment of the present invention can execute a key information extraction method provided in any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing a key information extraction method.

[0128] Embodiment 5

[0129] According to an embodiment of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0130] Figure 5 A schematic diagram of the structure of an electronic device applicable to the key information extraction method in the embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0131] like Figure 5 As shown, the electronic device includes: one or more processors 510, a memory 520, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides part of the necessary operations (for example, as a device array, a group of blade devices, or a multi-processor system). Figure 5 A processor 510 is taken as an example.

[0132] The memory 520 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform a key information extraction method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform a key information extraction method provided in the present application.

[0133] The memory 520 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to a key information extraction method in an embodiment of the present application (for example, the attached Figure 4 The processor 510 executes various functional applications and data processing of the electronic device by running the non-transient software programs, instructions and modules stored in the memory 520, that is, a key information extraction method in the above method embodiment is implemented.

[0134] The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of an electronic device that implements a key information extraction method, etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 520 may optionally include a memory remotely arranged relative to the processor 510, and these remote memories may be connected to an electronic device that performs a key information extraction method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The electronic device for executing a key information extraction method may further include: an input device 530 and an output device 540. The processor 510, the memory 520, the input device 530 and the output device 540 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0136] The input device 530 can receive input digital or character information, and generate key signal input related to user settings and function control of an electronic device that performs a key information extraction method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, and a joystick. The output device 540 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen. Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0140] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0142] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

[0143] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A key information extraction method, characterized in that: The method comprises: Performing content analysis on the target text to obtain analysis results; wherein the analysis results include: text content; Performing regular expression matching on the text content and the set field and obtaining a regular expression matching result; wherein the set field is a field used to characterize the location of key information; wherein the regular expression matching is a string matching method used to check whether the text contains the set field; Determine a method for extracting key information according to the regular matching result; wherein the extraction method includes a first extraction method and a second extraction method; If the extraction method is determined to be the first extraction method, a key information block is determined, and key information is extracted from the key information block; wherein the key information block is the paragraph identified by the summary field or the keyword field; If the extraction method is determined to be the second extraction method, inputting the target text into a language analysis model; extracting key information of the target text through the language analysis model; The extracting of key information of the target text through the language analysis model includes: performing word segmentation processing on the target text to obtain a word segmentation result; inputting the word segmentation result into the language analysis model so that the language analysis model can establish a word vector matrix for the word segmentation result; obtaining the contribution of words in the word segmentation result to the target text according to the word vector matrix; determining key information according to the contribution of the words, and extracting the key information; The contribution of the words in the word segmentation result to the target text refers to the similarity obtained by calculating the distance between the word vector in the word vector matrix and the cluster center; the cluster center is obtained by clustering the word vectors in the word vector matrix established by the language analysis model for the word segmentation result; before inputting the target text into the language analysis model, it also includes: counting the number of characters in the target text; determining the language analysis model version according to the number of characters in the target text, wherein the language analysis model version includes: a complete version of the language analysis model and a simplified version of the language analysis model; The complete language analysis model is trained by using the word segmentation results of the entire target document as a data set; the simplified language analysis model is trained by using the word segmentation results of pre-selected paragraphs in the target document as a data set; The step of performing regular expression matching on the text content and the set field and obtaining a regular expression matching result includes: Perform regular matching on the text title and the directory field; wherein the directory field refers to a directory field having a title font; Perform regular expression matching on the body text before the directory field, the summary field and the keyword field, and obtain a regular expression matching result; Wherein, before parsing the target text to obtain the parsing result, the method further includes: Receiving input text and checking the format of the input text; wherein checking the format of the input text is to check whether the format of the input text can be parsed into the format of a plain text file; If the format of the input text meets the set requirements, the input text is divided into target text.

2. The method according to claim 1, characterized in that Determining a method for extracting key information according to the regular matching result includes: Establishing an association relationship between the regular matching result and the method for extracting the key information; If the regular expression is matched successfully, the first extraction method of the key information is determined; If the regular matching is unsuccessful, the key information is extracted in the second extraction method.

3. The method according to claim 1, characterized in that The target text is parsed to obtain the parsing result, including: Convert the target text into a preset format to obtain a format conversion result; A file in a target format is selected from the format conversion result as the parsing result.

4. The method according to claim 1, characterized in that: The determining key information according to the contribution of the words and extracting the key information includes: Sorting the words according to their contribution; Extracting the words ranked within a set range as key words; and / or, Linearly combining the words to obtain a sentence, and calculating the contribution of the sentence according to the contribution of the words; The sentences are sorted, and the sentences ranked within a set range are extracted as key sentences.

5. A key information extraction device, characterized in that: The device comprises: The parsing module is used to parse the target text to obtain a parsing result; wherein the parsing result includes: text content; A regular matching module, used for performing regular matching between the text content and the set field and obtaining a regular matching result; wherein the regular matching is a string matching method used for checking whether the text contains the set field; A key information extraction module, used to determine a key information extraction method according to the regular matching result; wherein the extraction method includes a first extraction method and a second extraction method; The key information extraction module includes: a first key information extraction submodule, which is used to determine a key information block and extract key information from the key information block if the extraction method is determined to be the first extraction method; wherein the key information block is a paragraph identified by a summary field or a keyword field; The key information extraction module includes: a target text input submodule, which is used to input the target text into the language analysis model if the extraction method is determined to be the second extraction method; a second key information extraction submodule, which is used to extract the key information of the target text through the language analysis model; Among them, the second key information extraction submodule includes: a word segmentation unit, which is used to perform word segmentation processing on the target text to obtain a word segmentation result; a word segmentation result input unit, which is used to input the word segmentation result into the language analysis model so that the language analysis model can establish a word vector matrix for the word segmentation result; a contribution analysis unit, which is used to obtain the contribution of the words in the word segmentation result to the target text according to the word vector matrix; a key information extraction unit, which is used to determine key information according to the contribution of the words and extract the key information; The contribution of the words in the word segmentation result to the target text refers to the similarity obtained by calculating the distance between the word vector in the word vector matrix and the cluster center; the cluster center is obtained by clustering the word vectors in the word vector matrix established by the language analysis model for the word segmentation result; The device further comprises: a character number counting module, which is used to count the number of characters of the target text before inputting the target text into the language analysis model; a language analysis model version determination module, which is used to determine the language analysis model version according to the number of characters of the target text, wherein the language analysis model version includes: a complete version language analysis model and a simplified version language analysis model; The complete language analysis model is trained by using the word segmentation results of the entire target document as a data set; the simplified language analysis model is trained by using the word segmentation results of pre-selected paragraphs in the target document as a data set; Wherein, the regular matching module includes: A first regular matching submodule is used to perform regular matching between the text title and the directory field; wherein the directory field refers to a directory field having a title font; A second regular expression matching submodule is used to perform regular expression matching on the body text before the directory field, the summary field and the keyword field, and obtain a regular expression matching result; Wherein, the device further comprises: A text format checking module, used for receiving an input text and checking the format of the input text before parsing the target text to obtain the parsing result; wherein the checking the format of the input text is to check whether the format of the input text can be parsed into the format of a plain text file; The target text segmentation module is used to segment the input text into target text if the format of the input text meets the set requirements.

6. An electronic device, characterized in that: The device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a key information extraction method as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute a key information extraction method as described in any one of claims 1-4.

Images