A web-based information extraction method and apparatus

Abstract

The application discloses a webpage-based information extraction method and device. The extraction method obtains a text extraction template, obtains a keyword of to-be-extracted information as an extraction word, and then perfects corresponding columns in the text extraction template through associated words of the extraction word to obtain a target text template. The type of the to-be-extracted information can be determined through a label theme of the target text template and a preset text density, the target area on the target webpage is accurately determined, the word field of the target area is extracted through the associated words, and the extraction word and the word field are correspondingly displayed in a preset area of the target text template, so that the key target information in the webpage on the network can be accurately obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text extraction methods, and in particular to a webpage-based information extraction method and device. BACKGROUND

[0002] With the rapid development of the Internet in recent years, various information disclosures are mostly transmitted through the Internet as a medium. For documents that need to be disclosed according to a specified time and manner, they are usually published through web pages in a fixed format. For example, prospectuses of listed companies, various financial reports and interim reports. Such documents mostly have a long length or various financial data, and it takes a lot of effort and time to browse the entire document, which is not conducive to quickly obtaining the key information disclosure of each listed company.

[0003] Therefore, how to accurately obtain the key target information in the web page on the network is a technical problem to be solved at present. SUMMARY

[0004] The webpage-based information extraction method and device provided by the present application can accurately obtain the key target information in the web page on the network.

[0005] The embodiments of the present application provide the following solutions.

[0006] In a first aspect, the embodiments of the present application provide a webpage-based information extraction method, which comprises the following steps.

[0007] Obtaining an extraction word and a text extraction template, wherein the extraction word is a key word of the information to be extracted;

[0008] Obtaining a target text template according to the associated vocabulary of the extraction word and the text extraction template;

[0009] Determining a target area of a target web page according to the label theme of the target text template and a preset text density, wherein the label theme is the file name of the information to be extracted;

[0010] Extracting a vocabulary field of the target area according to the associated vocabulary, and displaying the extraction word and the vocabulary field in a preset area of the target text template.

[0011] In an optional embodiment, the step of obtaining a target text template according to the associated vocabulary of the extraction word and the text extraction template comprises the following steps.

[0012] Obtaining a vocabulary heat collection according to the heat index of each extended word of the extraction word;

[0013] Determining the associated vocabulary according to the extended word greater than a preset heat in the vocabulary heat collection;

[0014] inputting the associated vocabulary into the text extraction template to obtain the target text template.

[0015] In an optional embodiment, the extracting the vocabulary field of the target region according to the associated vocabulary comprises:

[0016] determining a vocabulary extraction interval according to the associated vocabulary and a preset target character;

[0017] determining a target position in the target region according to the vocabulary extraction interval;

[0018] obtaining the vocabulary field according to text information of the target position.

[0019] In an optional embodiment, before the determining the target position in the target region according to the vocabulary extraction interval, the method further comprises:

[0020] updating the vocabulary extraction interval according to a preset character length.

[0021] In an optional embodiment, the label theme is a financial statement, and after the extracting the target field of the target region according to the associated vocabulary and correspondingly displaying the extracted vocabulary and the target field in a preset area of the target text template, the method further comprises:

[0022] verifying the vocabulary field according to a preset cross-checking relationship and correspondingly displaying a verification result.

[0023] In an optional embodiment, after the determining the target region of the target webpage according to the label theme of the target text template and a preset text density, the method further comprises:

[0024] determining a label page number according to the extracted vocabulary and the label theme;

[0025] updating the target region according to the label page number.

[0026] In an optional embodiment, after the correspondingly displaying the extracted vocabulary and the vocabulary field in the preset area of the target text template, the method further comprises:

[0027] obtaining a display information set according to each display result of the preset area in a preset time period;

[0028] processing the display information set according to a preset sorting rule to obtain target display information.

[0029] In a second aspect, the embodiments of the present application further provide a webpage-based information extraction device, which comprises:

[0030] The first obtaining module is configured to obtain an extraction word and a text extraction template, wherein the extraction word is a keyword of the information to be extracted.

[0031] The first obtaining module is configured to obtain a target text template according to the associated vocabulary of the extraction word and the text extraction template.

[0032] The first determining module is configured to determine a target area of the target webpage according to a label theme of the target text template and a preset text density, wherein the label theme is a file name of the information to be extracted.

[0033] The first display module is configured to extract a vocabulary field of the target area according to the associated vocabulary, and display the extraction word and the vocabulary field in a preset area of the target text template.

[0034] In a third aspect, an electronic device is provided, including a processor and a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the electronic device to perform the steps of any of the methods of the first aspect.

[0035] In a fourth aspect, a computer-readable storage medium is provided, having stored thereon a computer program that, when executed by a processor, implements the steps of any of the methods of the first aspect.

[0036] The information extraction method and device based on a webpage provided by the present application have the following advantages compared with the prior art.

[0037] The present application obtains a text extraction template, obtains a keyword of the information to be extracted as an extraction word, and then perfects the corresponding column in the text extraction template through the associated vocabulary of the extraction word to obtain a target text template. The type of the information to be extracted can be determined through the label theme of the target text template and the preset text density, the target area can be accurately determined on the target webpage, the vocabulary field of the target area can be extracted through the associated vocabulary, and the extraction word and the vocabulary field can be displayed in the preset area of the target text template, so that the key target information in the webpage on the network can be accurately obtained. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0039] Figure 1A flow chart of a webpage-based information extraction method provided by an embodiment of the present application;

[0040] Figure 2 A structural schematic diagram of a webpage-based information extraction device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the embodiments of the present application.

[0042] Please refer to Figure 1 , Figure 1 A flow chart of a webpage-based information extraction method provided by an embodiment of the present application, the method comprising:

[0043] S11, obtaining an extraction word and a text extraction template, wherein the extraction word is a keyword of information to be extracted.

[0044] Specifically, the extraction word can be obtained according to the keyword of the information to be extracted. For example, in the prospectus, the capital issued by the company needs to be extracted, and it can be confirmed that the extraction word is “issue”. The text extraction template can be formulated according to the information to be extracted. If more information needs to be extracted at a time, multiple columns can be formulated to facilitate human-computer interaction. After obtaining the extraction word and the text extraction template, step S12 is entered.

[0045] S12, obtaining a target text template according to the associated vocabulary of the extraction word and the text extraction template.

[0046] Specifically, the associated vocabulary is a vocabulary with the same meaning as the extraction word, for example, the extraction word is “laser”, and the associated vocabulary can include “laser”. The expansion mode can be realized according to the keyword expansion tool, for example, the commonly used seo keyword tool, the associated vocabulary and the extraction word are input into the text extraction template, and the target text template is obtained.

[0047] In a specific embodiment, obtaining a target text template according to the associated vocabulary of the extraction word and the text extraction template comprises:

[0048] obtaining a vocabulary heat collection according to the heat index of each expansion word of the extraction word; determining the associated vocabulary according to the expansion word greater than the preset heat in the vocabulary heat collection; inputting the associated vocabulary into the text extraction template to obtain the target text template.

[0049] Specifically, there can be many associated words in the extracted word, and if all of them correspond to the input text extraction template, the text extraction template needs to be set with more columns, but the use of similar prospectuses is relatively fixed, and the use probability of some expanded associated words is very small, causing redundant calculation. The heat index represents the use frequency of each expanded word in this type of file. By extracting the heat index of each expanded word of the word, the use frequency distribution of each expanded word can be known. The expanded word in the word heat set that is greater than the preset heat is an expanded word with a higher use frequency distribution. The preset heat can be set according to the actual situation to determine the associated word with a higher use frequency. The associated word is input into the text extraction template to obtain the target text template with accurate and simplified information. After obtaining the target text template, step S13 is entered.

[0050] S13, determining a target area of the target webpage according to a label topic of the target text template and a preset text density, wherein the label topic is a file name of the information to be extracted.

[0051] Specifically, in the webpage of the browser, the information of the webpage is arranged in an HTML (Hyper Text Markup Language, Hyper Text Markup Language) document, and the distribution of the text in the entire webpage has certain differences according to the webpage format. For example, when a prospectus is displayed on a webpage, the text of the prospectus is relatively dense compared with other content on the webpage. In order to accurately determine the specific area of the file in the webpage, the type of the file can be determined according to the file name, different types correspond to different text densities, and thus the target area of the target webpage corresponding to the display file can be determined. It should be noted that the file to be extracted usually has a fixed format, so there is a corresponding text density, for example, the text density of the prospectus is larger than that of the financial report. After determining the target area, step S14 is entered.

[0052] S14, extracting a word field of the target area according to the associated word, and displaying the extracted word and the word field corresponding to the target text template in a preset area.

[0053] Specifically, still taking the prospectus as an example, the associated word exists in the word field, which means that the word field includes the associated word and may have key information that needs to be extracted. Therefore, the word field is extracted and displayed in the preset area of the target text template, and the extracted word and the word field are displayed corresponding to the target text template, so as to accurately obtain the key target information in the webpage on the network. The preset area can be a column of the target text template corresponding to the extracted word, and the associated word in the word field can be highlighted in the column, for example, highlighted, to prompt the reader to pay attention.

[0054] In a specific embodiment, the vocabulary field of the target region is extracted according to the associated vocabulary, comprising:

[0055] The vocabulary extraction interval is determined according to the associated vocabulary and the preset target character, the target position in the target region is determined according to the vocabulary extraction interval, and the vocabulary field is obtained according to the text information of the target position.

[0056] Specifically, the target character is a length mark of the sentence with the associated vocabulary, which can be a comma "," or a period "." in the prospectus. The vocabulary extraction interval can be determined according to the associated vocabulary and the preset target character. The text information of the target position is extracted according to the further determined target position, and the vocabulary field is obtained.

[0057] For example, a prospectus discloses the following content: In the past ten years, a large number of Internet users have been accumulated by providing Internet news information content and other interactive services, but the difference in user access volume with other commercial websites is still large. According to the authoritative statistics, on August 22, 2011, the number of independent visitors operated by the company was 19608 per million, the number of independent visitors of the global network was 9703 per million, and the number of independent visitors of other commercial websites was 247343 per million. In addition, with the opening of the Internet information service industry and sufficient financial support, other commercial websites can not only provide news information, but also can flexibly use the advantages of comprehensive services to accumulate users, which poses a great challenge to the company.

[0058] The associated vocabulary is "independent visitor", the target character is preset as a period ".", the determined vocabulary extraction interval is "according to the authoritative statistics, on August 22, 2011, the number of independent visitors operated by the company was 19608 per million, the number of independent visitors of the global network was 9703 per million, and the number of independent visitors of other commercial websites was 247343 per million.", the target position of the target region can be determined through the vocabulary extraction interval, and the vocabulary field to be extracted can be obtained.

[0059] In specific implementation, since the text on the target webpage is mostly written and edited by hand, there may be problems of non-standard character use. Only by determining the vocabulary extraction interval through the target character, the final obtained vocabulary field and the required vocabulary field have a certain deviation.

[0060] To solve the above problem, in a specific embodiment, before determining the target position in the target region according to the vocabulary extraction interval, further comprising:

[0061] The vocabulary extraction interval is updated according to the preset character length.

[0062] Specifically, the character length can be freely defined, for example, it can be the position of 30-50 fields before and after the associated vocabulary. After updating the vocabulary extraction interval by the character length, the character length can be set according to the information actually needed to be extracted, and then the vocabulary field can be accurately extracted, and the omission of key information is reduced.

[0063] In a specific application, the disclosure of information through the webpage is diversified, for example, the prospectus is disclosed in the form of text, and other information is disclosed in the form of table or graph, which may cause information extraction error.

[0064] In a specific embodiment, the label topic is a financial statement, the target field of the target area is extracted according to the associated vocabulary, and the extracted word and the target field are correspondingly displayed after the preset area of the target text template, further comprising:

[0065] The vocabulary field is verified according to the preset reconciliation relationship, and the verification result is correspondingly displayed.

[0066] Specifically, it can be understood that the reconciliation relationship of the financial statement refers to the relationship between the numbers in the account book and the financial statement, which can be used for mutual examination and verification. For example, the closing balance of each total classification account and the sum of the closing balances of each two-level account or detailed classification account to which it belongs have a mutual consistent and verifiable relationship. After extracting the target field of the target area according to the associated vocabulary, in order to further verify whether the extraction is accurate, the vocabulary field is verified through the preset reconciliation relationship. If the verification is passed, the verification result is correspondingly displayed as correct; otherwise, if the verification is not passed, the verification result is correspondingly displayed as incorrect.

[0067] The disclosure of information through the webpage is not updated in real time, and may be disclosed according to relevant regulations and standards at fixed time, for example, 3-6 months. The disclosed content may be more, and part of the file may reach hundreds of pages, even thousands of pages, including index and other non-key information, which may lead to too much non-key information in the extracted part.

[0068] In a specific embodiment, after determining the target area of the target webpage according to the label topic of the target text template and the preset text density, further comprising:

[0069] According to the extracted word and the label topic, the label page number is determined; and the target area is updated according to the label page number.

[0070] Specifically, the file type of the information to be extracted can be known through the label topic, and the position of the information to be extracted in different types of files is different. Taking the prospectus as an example, the major matters prompt in the prospectus appears after page 4-8, which involves the company's dividend distribution policy and cash dividend ratio regulation. If the extraction word is dividend distribution, the label page number can be determined according to the extraction word and the label topic, and the target area is updated.

[0071] In a specific embodiment, after the extracted word and the vocabulary field are correspondingly displayed in the preset area of the target text template, the method further comprises:

[0072] According to each display result of the preset area within the preset time period, a display information set is obtained; and the display information set is processed according to a preset sorting rule to obtain target display information.

[0073] Specifically, the preset sorting rule can be set according to the extracted word. For example, the extracted word is business amount. After the extracted information is displayed according to the extracted word, the time period for display can be selected, and descending arrangement can be performed to improve the readability of the display information set.

[0074] Based on the same inventive concept as the extraction method, the embodiments of the present application also provide a web page-based information extraction device. Please refer to Figure 2 , which comprises:

[0075] The first acquisition module 201 is configured to acquire an extracted word and a text extraction template, wherein the extracted word is a keyword of information to be extracted.

[0076] The first obtaining module 202 is configured to obtain a target text template according to the associated vocabulary of the extracted word and the text extraction template.

[0077] The first determining module 203 is configured to determine a target area of a target web page according to a label topic of the target text template and a preset text density, wherein the label topic is a file name of the information to be extracted.

[0078] The first display module 204 is configured to extract a vocabulary field of the target area according to the associated vocabulary, and correspondingly display the extracted word and the vocabulary field in a preset area of the target text template.

[0079] Based on the same inventive concept as the extraction method, the embodiments of the present application also provide an electronic device, which comprises a processor and a memory. The memory is coupled to the processor, and the memory stores instructions. When the instructions are executed by the processor, the electronic device performs the steps of any one of the extraction methods.

[0080] Based on the same inventive concept as the extraction method, the embodiments of the present application also provide a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, the steps of any one of the extraction methods are implemented.

[0081] The technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0082] By acquiring the text extraction template, a keyword of the information to be extracted is acquired as an extraction word, and then the corresponding column of the text extraction template is perfected through the associated vocabulary of the extraction word to obtain a target text template; through the label theme of the target text template and the preset text density, the type of the information to be extracted can be determined, the target area on the target webpage is accurately determined, then the vocabulary field of the target area is extracted through the associated vocabulary, and the extraction word and the vocabulary field are correspondingly displayed in the preset area of the target text template, and thus the key target information in the webpage on the network can be accurately obtained.

[0083] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (modules, systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device that implements the flow Figure 1 one or more flows and / or blocks Figure 1 devices that implement the functions specified in one or more blocks or flows.

[0084] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the flow Figure 1 one or more flows and / or blocks Figure 1 devices that implement the functions specified in one or more blocks or flows.

[0085] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 devices that implement the functions specified in one or more blocks or flows.

[0086] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they understand the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0087] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method of information extraction based on a web page, characterized by, The method comprises: acquiring an extraction word and a text extraction template, wherein the extraction word is a keyword of information to be extracted; obtaining a target text template according to an associated vocabulary of the extraction word and the text extraction template; determining a target area of a target webpage according to a label topic of the target text template and a preset text density, wherein the label topic is a file name of the information to be extracted; extracting a vocabulary field of the target area according to the associated vocabulary, and correspondingly displaying the extraction word and the vocabulary field in a preset area of the target text template.

2. The web page-based information extraction method according to claim 1, characterized by, The obtaining of the target text template according to the associated vocabulary of the extraction word and the text extraction template comprises: obtaining a vocabulary heat collection according to a heat index of each extended word of the extraction word; determining the associated vocabulary according to the extended word greater than a preset heat in the vocabulary heat collection; inputting the associated vocabulary into the text extraction template to obtain the target text template.

3. The web page-based information extraction method according to claim 1, characterized in that, The extraction of the vocabulary field of the target area according to the associated vocabulary comprises: determining a vocabulary extraction interval according to the associated vocabulary and a preset target character; determining a target position located in the target area according to the vocabulary extraction interval; obtaining the vocabulary field according to text information of the target position.

4. The web page-based information extraction method according to claim 3, characterized by, The determination of the target position located in the target area according to the vocabulary extraction interval further comprises: updating the vocabulary extraction interval according to a preset character length.

5. The web page-based information extraction method of claim 1, wherein, The label topic is a financial report, and the extraction of the target field of the target area according to the associated vocabulary and the correspondingly displaying of the extraction word and the target field in the preset area of the target text template further comprises: verifying the vocabulary field according to a preset cross-checking relationship, and correspondingly displaying a verification result.

6. The web page-based information extraction method of claim 1, wherein, The determination of the target area of the target webpage according to the label topic of the target text template and the preset text density further comprises: determining a label page number according to the extraction word and the label topic; updating the target area according to the label page number.

7. The web page-based information extraction method of claim 1, wherein, The correspondingly displaying of the extraction word and the vocabulary field in the preset area of the target text template further comprises: obtaining a display information collection according to each display result of the preset area in a preset time period; processing the display information collection according to a preset sorting rule to obtain target display information.

8. An information extraction apparatus based on a web page, characterized by comprising: The device comprises: a first acquisition module configured to acquire an extraction word and a text extraction template, wherein the extraction word is a keyword of information to be extracted; a first obtaining module configured to obtain a target text template according to an associated vocabulary of the extraction word and the text extraction template; a first determining module configured to determine a target area of a target webpage according to a label topic of the target text template and a preset text density, wherein the label topic is a file name of the information to be extracted; a first display module configured to extract a vocabulary field of the target area according to the associated vocabulary, and correspondingly display the extraction word and the vocabulary field in a preset area of the target text template.

9. An electronic device, comprising: An electronic device comprising a processor and a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the electronic device to perform the steps of the method of any of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, A program which, when executed by a processor, implements the steps of the method of any of claims 1-7.

Images