Feasibility study report intelligent analysis and information extraction method, system and equipment

Abstract

The present invention discloses a method, system and device for intelligent parsing and information extraction of a feasibility study report, which relates to the fields of natural language processing and machine learning technology. The method is to first identify each chapter title in the report text by using a regular expression or a syntactic analysis method based on natural language processing technology after reading the feasibility study report document to obtain the report text, and determine the title category of each chapter title according to the title hierarchy structure. Then, for any chapter title whose title category belongs to the key objects of report review, the corresponding chapter content is extracted from the report text, and key business information is extracted from the chapter content by combining the Word2Vec model, named entity recognition and semantic matching technology. Finally, each chapter title and the corresponding title category / and key business information are summarized and output for display. This method has the advantages of high efficiency, high accuracy, versatility, high intelligence and scalability of information extraction.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and machine learning technology, and specifically relates to a method, system and equipment for intelligent analysis and information extraction of feasibility study reports. Background Art

[0002] A feasibility study report is a document submitted for approval by decision makers and competent authorities before engaging in an economic activity (investment). It involves detailed investigation, research and analysis of various factors, including economics, technology, production, supply and marketing, as well as social environment and laws, to determine favorable and unfavorable factors, whether the project is feasible, and estimate the success rate, economic benefits and social effects.

[0003] With the increasing number and complexity of investment projects, the need for review of project feasibility studies is becoming increasingly urgent. However, traditional review methods, which primarily rely on manual review, are inefficient, subjective, and prone to omissions, and are no longer able to meet the needs of modern project approval.

[0004] Therefore, how to perform document analysis and information extraction on feasibility study reports based on automation technology, especially natural language processing (NLP) technology, in order to obtain information extraction results that are conducive to rapid review has become a topic that technical personnel in this field urgently need to study. Summary of the Invention

[0005] The purpose of the present invention is to provide a feasibility study report intelligent analysis and information extraction method, system, computer equipment, computer readable storage medium and computer program product to solve the problems of low efficiency, strong subjectivity and easy omission in traditional report review methods.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In the first aspect, a method for intelligent analysis and information extraction of a feasibility study report is provided, comprising:

[0008] Obtain feasibility study report documents;

[0009] Using an open source library to identify the document format of the feasibility study report document, and selecting an appropriate text reading method according to the document format identification result, to read the feasibility study report document to obtain the report text;

[0010] Identify and obtain the chapter titles in the report text by using regular expressions or syntax analysis based on natural language processing technology;

[0011] Determine the title category of each chapter title according to the title hierarchy structure;

[0012] For any chapter title whose title category belongs to the focus of report review, extract the corresponding chapter content from the report text;

[0013] By combining the Word2Vec model, named entity recognition technology, and semantic matching technology, key business information is extracted from the chapter content, wherein the key business information includes the project name, construction unit, investment scale, funding source, and / or implementation plan;

[0014] Summarize and output the various chapter titles and their corresponding title categories and the key business information.

[0015] Based on the above invention content, a new solution for automated document analysis and information extraction of feasibility study reports based on natural language processing technology is provided. That is, after reading the feasibility study report document to obtain the report text, the chapter titles in the report text are first identified through regular expressions or syntactic analysis based on natural language processing technology, and the title categories of each chapter title are determined according to the title hierarchy structure. Then, for any chapter title whose title category belongs to the key object of report review, the corresponding chapter content is extracted from the report text, and key business information is extracted from the chapter content by combining the Word2Vec model, named entity recognition technology and semantic matching technology. Finally, the chapter titles and the corresponding title categories / and key business information are summarized and output for display. In this way, not only can information extraction results that are conducive to rapid review be obtained, efficient structuring of the report can be achieved, and data support can be provided for automated review, but it also has the advantages of high efficiency, high accuracy, versatility, high intelligence and scalability of information extraction, which is convenient for practical application and promotion.

[0016] In one possible design, after reading the feasibility study report document to obtain the report text, the method further includes:

[0017] Regular expressions are used to remove redundant data, blank characters and redundant blank lines between adjacent paragraphs in the report text to obtain a report text after data cleaning.

[0018] In one possible design, after reading the feasibility study report document to obtain the report text, the method further includes:

[0019] The report text is subjected to part-of-speech tagging and syntactic analysis based on natural language processing technology to obtain a grammatically corrected report text with consistent grammatical structure and semantic relationship.

[0020] In a possible design, the title categories of the respective chapter titles are determined according to the title hierarchy structure, including:

[0021] For each chapter title, extract corresponding chapter level information from the report text according to the title hierarchy structure;

[0022] For any chapter title in the report text, the corresponding title category is determined based on the known structural relationship of the feasibility study report and the corresponding chapter level information, wherein the title categories include preface non-final title, preface final title, main body non-final title, main body final title, conclusion non-final title and conclusion final title.

[0023] In one possible design, after determining the title category of each chapter title according to the title hierarchy structure, the method further includes:

[0024] For each chapter title, the corresponding chapter content is extracted from the report text, and the chapter content is segmented to obtain a number of corresponding segmented words. The segmented words are then arranged in descending order according to word frequency to obtain a corresponding segmented word queue. Finally, the first N segmented words are extracted from the segmented word queue, and word vectors of the first N segmented words are generated through the Word2Vec model to obtain a corresponding keyword vector group, where N represents an integer greater than or equal to 10.

[0025] For each of the chapter titles, importing the corresponding keyword vector group into a content topic identification model pre-trained based on a second machine learning algorithm to obtain a corresponding content topic identification result and a confidence level of the content topic identification result;

[0026] Based on the content theme identification results of each chapter title, the display order of each chapter title is adjusted according to the preset report content review order, and for at least two chapter titles with the same content theme identification results, the display order of the at least two chapter titles is adjusted in descending order according to the confidence level, wherein the report content review order includes multiple content themes arranged in order.

[0027] In one possible design, the Word2Vec model, named entity recognition technology, and semantic matching technology are combined to extract key business information from the chapter content, including:

[0028] Performing word segmentation processing on the chapter content to obtain multiple word segments;

[0029] Generate word vectors for the multiple word segments using a Word2Vec model;

[0030] Applying a K-means clustering algorithm to cluster the word vectors of the multiple word segments to obtain multiple cluster centers;

[0031] For each word in the multiple word segments, according to the multiple cluster centers and the corresponding word vectors, calculate the corresponding Euler distance to the multiple cluster centers, and use the corresponding word as the belonging word of a cluster center among the multiple cluster centers and corresponding to the shortest Euler distance;

[0032] For each of the multiple cluster centers, arrange all corresponding attribution words in descending order according to word frequency to obtain a corresponding attribution word queue, then extract the top M attribution words from the attribution word queue, and finally import the word vectors of the top M attribution words into a semantic recognition model pre-trained based on the first machine learning algorithm to obtain a corresponding semantic recognition result, where M represents an integer greater than or equal to 10;

[0033] For each attribute word of the first cluster center of the named entity semantics whose semantic recognition result is a named entity class, named entity recognition technology is used to extract named entity information from the sentence to which the corresponding word belongs, wherein the named entity information includes the project name and / or the construction unit;

[0034] For each attribute word of the second cluster center whose semantic recognition result is the key review category semantics, semantic matching technology is used to extract key review information from the sentence to which the corresponding word belongs, wherein the key review information includes investment scale, funding source and / or implementation plan;

[0035] The named entity information and the review focus information are aggregated to obtain key business information extracted from the chapter content, wherein the key business information includes the project name, construction unit, investment scale, funding source and / or implementation plan.

[0036] Secondly, a feasibility study report intelligent parsing and information extraction system is provided, which includes a report document acquisition unit, a report text reading unit, a chapter title recognition unit, a title category determination unit, a chapter content extraction unit, a business information extraction unit, and an information summary display unit;

[0037] The report document acquisition unit is used to acquire a feasibility study report document;

[0038] The report text reading unit is communicatively connected to the report document acquiring unit, and is configured to use an open source library to identify the document format of the feasibility study report document, and select an adapted text reading method according to the document format identification result, to read the feasibility study report document to obtain the report text;

[0039] The chapter title recognition unit is communicatively connected to the report text reading unit and is used to recognize and obtain each chapter title in the report text by using a regular expression or a syntax analysis method based on natural language processing technology;

[0040] The title category determination unit is communicatively connected to the chapter title identification unit and is used to determine the title category of each chapter title according to the title hierarchy structure;

[0041] The chapter content extraction unit is communicatively connected to the report text reading unit and the title category determination unit, and is used to extract the corresponding chapter content from the report text for any chapter title whose title category belongs to the key objects of report review;

[0042] The business information extraction unit is communicatively connected to the chapter content extraction unit and is used to extract key business information from the chapter content by combining a Word2Vec model, named entity recognition technology, and semantic matching technology, wherein the key business information includes project name, construction unit, investment scale, funding source, and / or implementation plan;

[0043] The information summary and display unit is communicatively connected to the title category determination unit and the business information extraction unit, respectively, and is used to summarize and output the various chapter titles and the corresponding title categories and the key business information.

[0044] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the feasibility study report intelligent parsing and information extraction method as described in the first aspect or any possible design of the first aspect.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the feasibility study report intelligent parsing and information extraction method as described in the first aspect or any possible design of the first aspect is executed.

[0046] In a fifth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect or any possible design of the first aspect.

[0047] Beneficial effects of the above scheme:

[0048] (1) The present invention creatively provides a new solution for automated document analysis and information extraction of feasibility study reports based on natural language processing technology. That is, after reading the feasibility study report document to obtain the report text, the chapter titles in the report text are first identified by regular expressions or syntactic analysis based on natural language processing technology, and the title categories of each chapter title are determined according to the title hierarchy structure. Then, for any chapter title whose title category belongs to the key objects of report review, the corresponding chapter content is extracted from the report text, and key business information is extracted from the chapter content by combining the Word2Vec model, named entity recognition technology and semantic matching technology. Finally, each chapter title and the corresponding title category / and key business information are summarized and output for display. In this way, not only can information extraction results that are conducive to rapid review be obtained, the report can be efficiently structured and data support can be provided for automated review, but also the information extraction has the advantages of high efficiency, high accuracy, versatility, high intelligence and scalability, which is convenient for practical application and promotion.

[0049] (2) High efficiency, that is, this solution can efficiently extract information from documents of different formats, reduce manual intervention, and shorten document analysis time;

[0050] (3) High accuracy: This solution uses technologies such as natural language processing and semantic matching to accurately extract key business information from reports, ensuring the accuracy of analysis results;

[0051] (4) It is universal, that is, this solution is applicable to various types of investment project reports and can automatically adapt to documents of different formats and contents;

[0052] (5) Highly intelligent, that is, this solution uses machine learning / deep learning technology to continuously optimize the extraction model, improve extraction quality, and reduce errors;

[0053] (6) Scalability, that is, this solution has good scalability and can adapt to changes in different report templates, standards and policy orientations, and support document processing needs in different fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1A flowchart of the method for intelligent analysis and information extraction of a feasibility study report provided in an embodiment of the present application.

[0056] Figure 2 A schematic diagram of the structure of the feasibility study report intelligent analysis and information extraction system provided in an embodiment of the present application.

[0057] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0059] It should be understood that although the terms first, second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are merely used to distinguish one object from another. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0060] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0061] Example

[0062] like Figure 1As shown, the feasibility study report intelligent parsing and information extraction method provided in the first aspect of this embodiment can be executed by, but is not limited to, a computer device with certain computing resources, such as a cloud server, a personal computer (PC, a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), a smart phone, a personal digital assistant (PDA) or a wearable device. Figure 1 As shown, the feasibility study report intelligent analysis and information extraction method may include, but is not limited to, the following steps S1 to S7.

[0063] S1. Obtain feasibility study report document.

[0064] In step S1, the feasibility study report document is the subject of review. It can be, but is not limited to, uploaded by the report approver or report producer. Furthermore, the feasibility study report document can support a variety of file formats, such as PDF (Portable Document Format), Word, TXT, and / or XML (eXtensible Markup Language).

[0065] S2. Use an open source library to identify the document format of the feasibility study report document, and select an appropriate text reading method based on the document format identification result to read the feasibility study report document to obtain the report text.

[0066] In step S2, the open source library specifically includes, but is not limited to, existing Apache POI (a free, open-source, cross-platform Java API written in Java that provides Java programs with the ability to read and write Microsoft Office format files. POI stands for "Poor Obfuscation Implementation," meaning "simplified obfuscation implementation") or PDFBox (a pure Java class library designed for developers to read and create PDF documents). Furthermore, the text reading method also uses existing methods.

[0067] After step S2, in order to ensure that the report text is clean and neat, so as to facilitate subsequent content parsing and information extraction, preferably, after reading the feasibility study report document to obtain the report text, the method further includes but is not limited to the following steps: using regular expressions to remove redundant data, blank characters and redundant blank lines between adjacent paragraphs in the report text to obtain the report text after data cleaning. The regular expression (often abbreviated as regex or regexp) is a pattern tool for matching, searching and manipulating text, and realizes efficient string processing through specific grammatical rules; the core functions of the regular expression include: (1) pattern matching, i.e., identifying strings in a specific format (such as email addresses, phone numbers, etc.); (2) text search and replacement, i.e., quickly locating or batch modifying text fragments that meet the rules; (3) input validation, i.e., ensuring that user input meets the preset format (such as password complexity). Therefore, the regular expression can be routinely applied to achieve the removal of redundant data, blank characters and redundant blank lines. In addition, in order to ensure the grammatical structure and semantic consistency of the report text, so as to further facilitate subsequent content parsing and information extraction, preferably, after reading the feasibility study report document to obtain the report text, the method further includes but is not limited to the following steps: based on natural language processing technology, the report text is subjected to part-of-speech tagging and syntactic analysis to obtain a grammatically corrected report text with consistent grammatical structure and semantic relationship. The aforementioned part-of-speech tagging is a basic task in the field of NLP. Its main goal is to determine the part of speech of each word in a sentence (such as noun, pronoun, adjective, adverb, verb, numeral, article, preposition, conjunction and interjection) to help the computer understand the structure and meaning of the sentence, so as to better perform subsequent processing and analysis. The aforementioned syntactic analysis is another important task in the field of NLP. Its main goal is to determine the syntactic structure of a sentence (i.e., the grammatical relationship between words in a sentence, generally represented by a tree structure, also called a syntax tree or grammar tree) to help the computer understand the structure and meaning of the sentence, so as to better perform subsequent processing and analysis. In this way, the part-of-speech tagging results and syntactic analysis results can be routinely obtained, and based on these results, it can be routinely judged whether the grammatical structure and semantic relationship of the report text are consistent. If not, it will be modified in a conventional way to ensure that a grammatically corrected report text with consistent grammatical structure and semantic relationship is finally obtained.

[0068] S3. Identify the chapter titles in the report text through regular expressions or syntactic analysis based on natural language processing technology.

[0069] In step S3, because the regular expression has text matching and search capabilities, the regular expression can be used to routinely identify the various chapter titles in the report text that contain or follow special characters such as "1.1," "1.2," or "1.1.3." Furthermore, because the syntactic analysis method can help a computer understand the structure and meaning of a sentence, the various chapter titles in the report text can also be routinely identified based on the sentence structure and meaning understanding results.

[0070] S4. Determine the title category of each chapter title according to the title hierarchy structure.

[0071] In step S4, the structural relationship of the feasibility study report is considered to mainly include three parts: subject, main text and appendix. Among them, the main text usually includes three parts: preface (which is used to briefly introduce the project background, basis, purpose and economic benefits, and explain the scope and requirements of the feasibility study), main body (which is used to describe in detail the market survey, scale and plan analysis, technical strength and level description, funding source analysis, economic benefit analysis, etc.) and conclusion (which is used to summarize and generalize the content of the entire report). The main text is the core area of ​​the report review, especially the project name, construction unit, investment scale, funding source and implementation plan. At the same time, it is also considered that there are multiple small chapters under a large chapter, and the detailed content of the main text will only be in the last chapter. Therefore, in order to quickly obtain information extraction results that are conducive to rapid review, it is necessary to first classify each chapter title (that is, identify whether the corresponding chapter title is a non-final title of the preface, a final title of the preface, a non-final title of the main body, a final title of the main body, a non-final title of the conclusion, or a final title of the conclusion, etc.), so that targeted information can be extracted based on the classification results, such as extracting key business information from the chapter content of the final title of the main body. In order to achieve the purpose of accurately classifying each chapter title, it is preferred to determine the title category of each chapter title according to the title hierarchy structure, including but not limited to the following steps S41 to S42.

[0072] S41. For each chapter title, extract corresponding chapter level information from the report text according to the title hierarchy structure.

[0073] In step S41, the title hierarchy structure can be, but is not limited to, composed of serial numbers such as "one", "two" and "three", or composed of prefixes such as "Chapter One", "Chapter Two" and "Chapter Three", so the chapter level information of the chapter titles can be routinely extracted based on these structures.

[0074] S42. For any chapter title in the report text, determine the corresponding title category based on the known structural relationship of the feasibility study report and the corresponding chapter level information, wherein the title categories include but are not limited to preface non-final title, preface final title, main body non-final title, main body final title, conclusion non-final title and conclusion final title, etc.

[0075] In step S42, the known structural relationship includes three parts: subject, text and appendix, among which the text part includes three parts: preface, main body and conclusion, and so on; therefore, based on this structural relationship and the corresponding chapter level information, it can be conventionally determined whether any chapter title is a preface non-last-level title, a preface last-level title, a main body non-last-level title, a main body last-level title, a conclusion non-last-level title or a conclusion last-level title, etc.

[0076] S5. For any chapter title whose title category belongs to the focus of report review, the corresponding chapter content is extracted from the report text.

[0077] In step S5, the report review focus objects can be pre-designated by the report approver according to actual review requirements, for example, the last level title of the preface and the last level title of the main body can be designated as the report review focus objects respectively.

[0078] S6. Combine the Word2Vec model, named entity recognition technology and semantic matching technology to extract key business information from the content of the chapter, where the key business information includes but is not limited to the project name, construction unit, investment scale, funding source and / or implementation plan.

[0079] In step S6, the Word2Vec model is an existing model that maps vocabulary into high-dimensional vectors through a neural network, including two architectures, CBOW and Skip-gram, for capturing the semantic relationship between words; the named entity recognition (NER) technology is an important technology in NLP, which aims to identify name entities in text, such as names of people, places, organizations or dates, and can therefore be used to extract key business information such as project names and / or construction units; the semantic matching technology is an important task in NLP, which aims to determine the semantic similarity or matching degree between two texts. It is widely used in various NLP tasks, such as information retrieval, question-answering systems, question repetition, dialogue systems and machine translation, and can therefore be used to extract key business information such as investment scale, funding sources and / or implementation plans. Considering that the chapter content may contain a large number of sentences, and each sentence does not necessarily contain key business information such as project name, construction unit, investment scale, funding source and / or implementation plan, in order to improve the extraction accuracy and extraction speed of key business information, preferably, the Word2Vec model, named entity recognition technology and semantic matching technology are combined to extract key business information from the chapter content, including but not limited to the following steps S61 to S68.

[0080] S61. Perform word segmentation processing on the chapter content to obtain multiple word segments.

[0081] In step S61, the specific process of the word segmentation processing can be implemented by, but is not limited to, using the jieba word segmentation tool.

[0082] S62. Generate word vectors for the multiple word segments using a Word2Vec model.

[0083] S63. Based on the word vectors of the multiple word segments, apply the K-means clustering algorithm to obtain multiple cluster centers.

[0084] In step S63, the K-means clustering algorithm is an iterative clustering analysis algorithm, and its steps are generally as follows: pre-dividing the data into K groups, and randomly selecting K objects as initial cluster centers, then calculating the distance between each object and each seed cluster center, assigning each object to the cluster center closest to it, and continuously updating the iterative cluster centers until the sum of squared errors (i.e., the criterion function) converges to obtain the clustering result. Considering that the existing K-means clustering algorithm may change the compactness and discreteness of the clusters due to the influence of extreme values ​​during the clustering process, thereby reducing the accuracy of the entire clustering result, preferably, based on the word vectors of the multiple word segments, the K-means clustering algorithm is applied to cluster to obtain multiple cluster centers, including but not limited to the following steps S631 to S635.

[0085] S631. For each word in the multiple word segmentations, based on the corresponding word vector and the word vectors of each other word in the multiple word segmentations, the corresponding Euler distance with each other word segmentation is calculated, and the number of distances whose corresponding Euler distance is less than a preset distance threshold is counted.

[0086] In step S631, the specific calculation formula of the Euler distance is the existing formula. For example, for a certain participle, if there are 628 other participles, 628 Euler distances will be calculated, and if 376 of the 628 Euler distances are less than the distance threshold, then the number of distances of the certain participle whose Euler distance is less than the preset distance threshold is 376. The aforementioned number of distances positively reflects the number of words in other participles that are located around the semantics of the certain participle in the multidimensional space, that is, it positively reflects the size of the neighbor density corresponding to the certain participle.

[0087] S632. According to the number of distances of each segmentation and the distribution position of each segmentation in the multidimensional space, determine multiple segmentations with maximum number of distances among the multiple segmentations, wherein each dimension in the multidimensional space corresponds one-to-one to each dimensional data value in the word vector.

[0088] In the step S632, the number of distances of each segmentation in the multiple segmentations is greater than the number of distances of adjacent segmentations in the multiple dimensions, which is similar to the method of finding extreme values ​​through partial derivatives of multivariate functions. According to the number of distances of each segmentation and the distribution position of each segmentation in the multidimensional space, the multiple segmentations with the maximum number of distances among the multiple segmentations are determined.

[0089] S633. Arrange the multiple segmentations in descending order according to the maximum value of the distance number to obtain a segmentation sequence.

[0090] S634. Select the first K word segmentations from the word segmentation sequence, and use the word vectors of the first K word segmentations as K initial cluster centers in a one-to-one correspondence, where K represents the total number of preset cluster centers.

[0091] S635. Based on the K initial cluster centers, apply the other steps of the K-means clustering algorithm and located after the initial cluster center selection step to cluster and obtain multiple cluster centers.

[0092] Therefore, based on the above steps S631 to S635, the word distribution information can be used to select multiple points with the highest density as the initial clustering centers, so as to effectively solve the problem of clustering falling into local optimality due to human interference or extreme value influence, so that the final clustering result can meet the conditions that the similarity of the same cluster is the highest and the similarity of different clusters is the lowest, thereby ensuring the stability of the clustering result.

[0093] S64. For each word in the multiple word segments, based on the multiple cluster centers and the corresponding word vectors, the corresponding Euler distance to the multiple cluster centers is calculated, and the corresponding word is used as the belonging word of a cluster center among the multiple cluster centers and corresponding to the shortest Euler distance.

[0094] In the step S64, for example, if the Euler distances of the certain word to the multiple cluster centers and the word vectors of the certain word are calculated respectively, and the Euler distances to the multiple cluster centers are: the Euler distance to cluster center A is 12 unit distances, the Euler distance to cluster center B is 34 unit distances; the Euler distance to cluster center C is 8 unit distances, the Euler distance to cluster center D is 21 unit distances, and the Euler distance to cluster center E is 17 unit distances, then the certain word can be used as the belonging word of cluster center C.

[0095] S65. For each cluster center among the multiple cluster centers, arrange all corresponding attribution words in order from high to low according to word frequency to obtain a corresponding attribution word queue, and then extract the first M attribution words from the attribution word queue, and finally import the word vectors of the first M attribution words into the semantic recognition model pre-trained based on the first machine learning algorithm to obtain the corresponding semantic recognition result, where M represents an integer greater than or equal to 10.

[0096] In step S65, M is, for example but not limited to, 12. A machine learning algorithm is a core artificial intelligence algorithm that specifically studies how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. It is the fundamental way to make computers intelligent. Specifically, the first machine learning algorithm preferably, but not limited to, uses a machine learning algorithm based on a graph neural network, a support vector machine, a K-nearest neighbor method, a stochastic gradient descent method, a multivariate linear regression method, a multilayer perceptron, a decision tree, a backpropagation neural network, or a radial basis function network, so as to quickly and accurately identify patterns in data. Thus, based on a certain amount of sample data (i.e., the model input is the word vectors of M segmented words, and the model output is the semantic labels corresponding to the M segmented words), a validated semantic recognition model can be trained using conventional calibration and verification modeling methods (specifically, the process includes a calibration process and a verification process, i.e., first comparing the model simulation results with the measured data and then adjusting the model parameters based on the comparison results to ensure that the simulation results are consistent with the actual data). In addition, the semantic recognition results may include, but are not limited to, named entity semantics, review focus semantics, or other semantics, and different semantic labels may be pre-specified according to review requirements.

[0097] S66. For each belonging word of the first cluster center whose semantic recognition result is the named entity class semantics, use the named entity recognition technology to extract the named entity information from the sentence to which the corresponding word belongs, wherein the named entity information includes but is not limited to the project name and / or construction unit, etc.

[0098] In step S66, since the named entity recognition technology is an existing technology, the specific extraction process of the named entity information can be conventionally derived based on existing technical means.

[0099] S67. For each belonging word of the second cluster center whose semantic recognition result is the review focus semantics, use semantic matching technology to extract the review focus information from the sentence to which the corresponding word belongs, where the review focus information includes but is not limited to investment scale, source of funds and / or implementation plan, etc.

[0100] In step S67, since the semantic matching technology is an existing technology, the specific extraction process of the review focus information can be conventionally derived based on existing technical means.

[0101] S68. Summarize the named entity information and the review focus information to obtain key business information extracted from the chapter content, wherein the key business information includes but is not limited to the project name, construction unit, investment scale, funding source and / or implementation plan, etc.

[0102] Therefore, based on the aforementioned steps S61 to S68, key business information can be extracted only for sentences containing key review semantic words and key review semantic words, without the need to extract information from each sentence. Therefore, the extraction purpose can be achieved in a targeted manner, which is conducive to improving the extraction accuracy and speed of key business information.

[0103] S7. Summarize and output the chapter titles and their corresponding title categories and the key business information.

[0104] In step S7, since this embodiment will not extract key business information for chapter titles whose title categories do not belong to the focus of report review, for such chapter titles, only the corresponding title categories are displayed, and for chapter titles whose title categories belong to the focus of report review, the corresponding title categories and the key business information will be displayed. In addition, considering that the order in which the titles of the feasibility study report are listed may be inconsistent with the order in which the report content is reviewed, in order to make the display order of the various chapter titles consistent with the order in which the report content is reviewed, so as to further facilitate the report approver to quickly review the information extraction results, preferably, after determining the title categories of the various chapter titles according to the title hierarchy structure, the method also includes but is not limited to the following steps S701 to S703.

[0105] S701. For each chapter title, the corresponding chapter content is extracted from the report text, and the chapter content is segmented to obtain a corresponding number of segmented words. The segmented words are then arranged in descending order according to word frequency to obtain a corresponding segmented word queue. Finally, the first N segmented words are extracted from the segmented word queue, and word vectors of the first N segmented words are generated through the Word2Vec model to obtain a corresponding keyword vector group, where N represents an integer greater than or equal to 10.

[0106] In step S701, N may be, for example but not limited to, 12. In addition, the keyword vector group includes the word vectors of the first N word segments.

[0107] S702. For each of the chapter titles, the corresponding keyword vector group is imported into a content topic recognition model pre-trained based on a second machine learning algorithm to obtain a corresponding content topic recognition result and a confidence level of the content topic recognition result.

[0108] In step S702, the second machine learning algorithm preferably employs, but is not limited to, a machine learning algorithm based on graph neural networks, support vector machines, K-nearest neighbor methods, stochastic gradient descent, multivariate linear regression, multilayer perceptrons, decision trees, backpropagation neural networks, or radial basis function networks, to quickly and accurately identify patterns in the data. Similarly, a content topic identification model can be trained and validated using conventional calibration and validation modeling methods based on a certain amount of sample data (i.e., the model input is the word vectors of N segmented words, and the model output is the content topic labels corresponding to the N segmented words). Furthermore, the content topic identification results may include, but are not limited to, topics such as "project background and objectives," "technical feasibility analysis," "market feasibility analysis," "economic feasibility analysis," "environmental and social impact assessment," or "risk assessment and response measures." Different content topic labels can be pre-assigned based on review requirements.

[0109] S703. Based on the content theme identification results of each chapter title, adjust the display order of each chapter title according to the preset report content review order, and for at least two chapter titles with the same content theme identification results, adjust the display order of the at least two chapter titles in descending order according to the confidence level, wherein the report content review order includes multiple content themes arranged in order.

[0110] In step S703, the order of reviewing the contents of the report can be exemplified as follows: the technical feasibility analysis theme, the market feasibility analysis theme, the economic feasibility analysis theme, the environmental and social impact assessment theme, the risk assessment and response measures theme, and the project background and objectives theme, etc., and the order of display of the chapter titles can be adjusted in this order. In addition, if two chapter titles both belong to the economic feasibility analysis theme: that is, the confidence level of Chapter A belonging to the economic feasibility analysis theme is 70%, and the confidence level of Chapter B belonging to the economic feasibility analysis theme is 80%, then the order of display of the two chapter titles can be determined as: Chapter B and Chapter A.

[0111] Therefore, based on the intelligent parsing and information extraction method of the feasibility study report described in the aforementioned steps S1 to S7, a new solution for automated document analysis and information extraction of feasibility study reports based on natural language processing technology is provided. That is, after reading the feasibility study report document to obtain the report text, the chapter titles in the report text are first identified through regular expressions or syntactic analysis based on natural language processing technology, and the title categories of each chapter title are determined according to the title hierarchy structure. Then, for any chapter title whose title category belongs to the key object of report review, the corresponding chapter content is extracted from the report text, and key business information is extracted from the chapter content by combining the Word2Vec model, named entity recognition technology and semantic matching technology. Finally, the chapter titles and the corresponding title categories / and key business information are summarized and output for display. In this way, not only can information extraction results that are conducive to rapid review be obtained, efficient structuring of the report can be achieved, and data support can be provided for automated review, but it also has the advantages of high efficiency, high accuracy, versatility, high intelligence and scalability of information extraction, which is convenient for practical application and promotion.

[0112] like Figure 2 As shown, the second aspect of this embodiment provides a virtual system for implementing the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect, comprising a report document acquisition unit, a report text reading unit, a chapter title recognition unit, a title category determination unit, a chapter content extraction unit, a business information extraction unit, and an information summary display unit;

[0113] The report document acquisition unit is used to acquire a feasibility study report document;

[0114] The report text reading unit is communicatively connected to the report document acquiring unit, and is configured to use an open source library to identify the document format of the feasibility study report document, and select an adapted text reading method according to the document format identification result, to read the feasibility study report document to obtain the report text;

[0115] The chapter title recognition unit is communicatively connected to the report text reading unit and is used to recognize and obtain each chapter title in the report text by using a regular expression or a syntax analysis method based on natural language processing technology;

[0116] The title category determination unit is communicatively connected to the chapter title identification unit and is used to determine the title category of each chapter title according to the title hierarchy structure;

[0117] The chapter content extraction unit is communicatively connected to the report text reading unit and the title category determination unit, and is used to extract the corresponding chapter content from the report text for any chapter title whose title category belongs to the key objects of report review;

[0118] The business information extraction unit is communicatively connected to the chapter content extraction unit and is used to extract key business information from the chapter content by combining a Word2Vec model, named entity recognition technology, and semantic matching technology, wherein the key business information includes project name, construction unit, investment scale, funding source, and / or implementation plan;

[0119] The information summary and display unit is communicatively connected to the title category determination unit and the business information extraction unit, respectively, and is used to summarize and output the various chapter titles and the corresponding title categories and the key business information.

[0120] The working process, working details and technical effects of the aforementioned system provided in the second aspect of this embodiment can be found in the feasibility study report intelligent analysis and information extraction method described in the first aspect, and will not be repeated here.

[0121] like Figure 3 As shown, the third aspect of this embodiment provides a computer device for executing the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect, comprising a memory, a processor, and a transceiver communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect. Specifically, the memory may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), flash memory, first-input first-output (FIFO), and / or first-input last-output (FILO) memory, etc.; the processor may include, but is not limited to, a microprocessor from the STM32F105 series. Furthermore, the computer device may include, but is not limited to, a power module, a display screen, and other necessary components.

[0122] The working process, working details and technical effects of the aforementioned computer equipment provided in the third aspect of this embodiment can be found in the feasibility study report intelligent analysis and information extraction method described in the first aspect, and will not be repeated here.

[0123] A fourth aspect of this embodiment provides a computer-readable storage medium storing instructions including the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect, that is, the computer-readable storage medium stores instructions that, when executed on a computer, execute the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect. The computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, computer-readable storage media such as a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device.

[0124] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be found in the feasibility study report intelligent analysis and information extraction method described in the first aspect, and will not be repeated here.

[0125] A fifth aspect of this embodiment provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements the method for intelligent parsing and information extraction of a feasibility study report as described in the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0126] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A feasibility study report intelligent analysis and information extraction method, characterized in that: include: Obtain feasibility study report documents; Using an open source library to identify the document format of the feasibility study report document, and selecting an appropriate text reading method according to the document format identification result, to read the feasibility study report document to obtain the report text; Identify and obtain the chapter titles in the report text by using regular expressions or syntax analysis based on natural language processing technology; Determine the title categories of the chapter titles according to the title hierarchy structure, wherein the title categories include preface non-final title, preface final title, main body non-final title, main body final title, conclusion non-final title and conclusion final title; For any chapter title whose title category belongs to the focus of report review, extract the corresponding chapter content from the report text, wherein the focus of report review is designated as the last-level title of the preface and the last-level title of the main body; Combined with the Word2Vec model, named entity recognition technology and semantic matching technology, key business information is extracted from the chapter content, specifically including: performing word segmentation processing on the chapter content to obtain multiple word segments; generating word vectors of the multiple word segments through the Word2Vec model; clustering the multiple word segments using the K-means clustering algorithm based on the word vectors of the multiple word segments to obtain multiple cluster centers; for each word in the multiple word segments, according to the multiple cluster centers and the corresponding word vectors, calculating the corresponding Euler distance to the multiple cluster centers, and using the corresponding word as the belonging word of a cluster center in the multiple cluster centers and corresponding to the shortest Euler distance; for each cluster center in the multiple cluster centers, arranging all corresponding belonging words in descending order according to word frequency to obtain a corresponding belonging word queue, and then extracting the top M belonging words from the belonging word queue, and finally extracting the top M belonging words from the belonging word queue. The word vector of the attributive word is imported into the semantic recognition model pre-trained based on the first machine learning algorithm to obtain the corresponding semantic recognition result, wherein M represents an integer greater than or equal to 10; for each attributive word of the first cluster center whose semantic recognition result is the named entity class semantics, the named entity recognition technology is used to extract the named entity information from the sentence to which the corresponding word belongs, wherein the named entity information includes the project name and the construction unit; for each attributive word of the second cluster center whose semantic recognition result is the review focus class semantics, the semantic matching technology is used to extract the review focus information from the sentence to which the corresponding word belongs, wherein the review focus information includes the investment scale, source of funds and implementation plan; the named entity information and the review focus information are summarized to obtain the key business information extracted from the content of the chapter, wherein the key business information includes the project name, construction unit, investment scale, source of funds and implementation plan; Summarize and output the various chapter titles and their corresponding title categories and the key business information.

2. The method for intelligent analysis and information extraction of a feasibility study report according to claim 1 is characterized in that: After reading the feasibility study report document to obtain the report text, the method further includes: Regular expressions are used to remove redundant data, blank characters and redundant blank lines between adjacent paragraphs in the report text to obtain a report text after data cleaning.

3. The method for intelligent analysis and information extraction of a feasibility study report according to claim 1 is characterized in that: After reading the feasibility study report document to obtain the report text, the method further includes: The report text is subjected to part-of-speech tagging and syntactic analysis based on natural language processing technology to obtain a grammatically corrected report text with consistent grammatical structure and semantic relationship.

4. The method for intelligent analysis and information extraction of a feasibility study report according to claim 1 is characterized in that: Determine the title category of each chapter title according to the title hierarchy structure, including: For each chapter title, extract corresponding chapter level information from the report text according to the title hierarchy structure; For any chapter title in the report text, the corresponding title category is determined based on the known structural relationship of the feasibility study report and the corresponding chapter level information.

5. The method for intelligent analysis and information extraction of a feasibility study report according to claim 1 is characterized in that: After determining the title categories of the respective chapter titles according to the title hierarchy structure, the method further includes: For each chapter title, the corresponding chapter content is extracted from the report text, and the chapter content is segmented to obtain a number of corresponding segmented words. The segmented words are then arranged in descending order according to word frequency to obtain a corresponding segmented word queue. Finally, the first N segmented words are extracted from the segmented word queue, and word vectors of the first N segmented words are generated through the Word2Vec model to obtain a corresponding keyword vector group, where N represents an integer greater than or equal to 10. For each of the chapter titles, importing the corresponding keyword vector group into a content topic identification model pre-trained based on a second machine learning algorithm to obtain a corresponding content topic identification result and a confidence level of the content topic identification result; Based on the content theme identification results of each chapter title, the display order of each chapter title is adjusted according to the preset report content review order, and for at least two chapter titles with the same content theme identification results, the display order of the at least two chapter titles is adjusted in descending order according to the confidence level, wherein the report content review order includes multiple content themes arranged in order.

6. A feasibility study report intelligent analysis and information extraction system, characterized by: It includes a report document acquisition unit, a report text reading unit, a chapter title recognition unit, a title category determination unit, a chapter content extraction unit, a business information extraction unit and an information summary display unit; The report document acquisition unit is used to acquire a feasibility study report document; The report text reading unit is communicatively connected to the report document acquiring unit, and is configured to use an open source library to identify the document format of the feasibility study report document, and select an adapted text reading method according to the document format identification result, to read the feasibility study report document to obtain the report text; The chapter title recognition unit is communicatively connected to the report text reading unit and is used to recognize and obtain each chapter title in the report text by using a regular expression or a syntax analysis method based on natural language processing technology; The title category determination unit is communicatively connected to the chapter title identification unit and is used to determine the title category of each chapter title according to the title hierarchy structure, wherein the title categories include preface non-final title, preface final title, main body non-final title, main body final title, conclusion non-final title and conclusion final title; The chapter content extraction unit is communicatively connected to the report text reading unit and the title category determination unit, and is used to extract the corresponding chapter content from the report text for any chapter title whose title category belongs to the report review focus object, wherein the report review focus object is designated as the last level title of the preface and the last level title of the main body; The business information extraction unit is communicatively connected to the chapter content extraction unit, and is used to extract key business information from the chapter content by combining the Word2Vec model, named entity recognition technology and semantic matching technology, specifically including: performing word segmentation processing on the chapter content to obtain multiple word segments; generating word vectors of the multiple word segments through the Word2Vec model; applying the K-means clustering algorithm to cluster the multiple word vectors to obtain multiple cluster centers; for each word in the multiple word segments, calculating the corresponding Euler distance to the multiple cluster centers based on the multiple cluster centers and the corresponding word vectors, and using the corresponding word as the belonging word of a cluster center in the multiple cluster centers and corresponding to the shortest Euler distance; for each cluster center in the multiple cluster centers, arranging all corresponding belonging words in descending order according to word frequency to obtain a corresponding belonging word queue, and then extracting from the belonging word queue The first M attributive words, and finally the word vectors of the first M attributive words are imported into the semantic recognition model pre-trained based on the first machine learning algorithm to obtain the corresponding semantic recognition results, wherein M represents an integer greater than or equal to 10; for each attributive word of the first cluster center whose semantic recognition result is the named entity class semantics, the named entity recognition technology is used to extract the named entity information from the sentence to which the corresponding word belongs, wherein the named entity information includes the project name and the construction unit; for each attributive word of the second cluster center whose semantic recognition result is the review focus class semantics, the semantic matching technology is used to extract the review focus information from the sentence to which the corresponding word belongs, wherein the review focus information includes the investment scale, source of funds and implementation plan; the named entity information and the review focus information are summarized to obtain the key business information extracted from the content of the chapter, wherein the key business information includes the project name, construction unit, investment scale, source of funds and implementation plan; The information summary and display unit is communicatively connected to the title category determination unit and the business information extraction unit, respectively, and is used to summarize and output the various chapter titles and the corresponding title categories and the key business information.

7. A computer device, characterized in that: It includes a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the feasibility study report intelligent analysis and information extraction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the feasibility study report intelligent parsing and information extraction method as described in any one of claims 1 to 5 is executed.

9. A computer program product comprising a computer program or instructions, characterized in that When executed by a computer, the computer program or the instruction implements the feasibility study report intelligent analysis and information extraction method as described in any one of claims 1 to 5.

Images