Automatic abstract extraction method and system based on latent semantic analysis
A technology of semantic analysis and automatic summarization, which is applied in the directions of semantic analysis, natural language data processing, and special data processing applications, etc. It can solve problems such as grammatical errors, logical incoherence, and blunt contextual cohesion in summaries, so as to reduce grammatical errors and reduce redundancy. Remaining information, the effect of expressing semantic coherence
Inactive Publication Date: 2017-10-20
成都数联铭品科技有限公司
View PDF4 Cites 17 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In the current automatic summarization technology, semantic information is not considered when calculating sentence scores based on TF-IDF technology. Words with the same meaning often have different TF-IDF values, which will lead to a large gap in the final score of sentences with the same meaning. , which in turn affects the quality of the abstract produced; the abstract generation algorithm based on the graph model regards sentences as nodes in the graph and the relationship between sentences as edges in the graph. Similarity is usually used to measure the relationship between sentences. However, the current Similarity measures are mostly based on literal rather than semantic similarity; generative summarization technology generates summaries of documents by using natural language processing technologies such as sentence fusion, sentence compression, and language generation, but current sentence fusion, sentence compression, and language generation related technologies Not yet mature enough to generate summaries with grammatical errors, incoherent logic, or poor context
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0023] The present invention will be further described in detail below in conjunction with test examples and specific embodiments. However, it should not be understood that the scope of the above subject matter of the present invention is limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.
[0024] The purpose of the present invention is to overcome the above-mentioned deficiencies existing in the prior art, and provide an automatic abstract extraction method based on latent semantic analysis. Co-occurrence information and semantic information, instead of simply selecting sentences based on word frequency or mutual "recommendation" between sentences, enables the generated summary to better reflect the topic expressed in the document.
[0025] In order to achieve the purpose of the above invention, the present invention provides the following technical solutions: an autom...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to the field of natural language processing, in particular to an automatic abstract extraction method and system based on latent semantic analysis. According to the method, a latent semantic analysis model is used when a sentence is extracted to generate an abstract, the latent semantic analysis model is constructed by using a larger corpus, the semantic similarity of a to-be-extracted text and a to-be-extracted semantic unit is calculated according to the model, co-occurrence information and semantic information of words in a document are fully considered, and sentence selection is carried out not simply based on mutual recommendation of word frequency or sequences so that the generated abstract can better reflect a theme expressed by the document. Meanwhile, compared with an abstract generation algorithm based on literal matching for similarity calculation or word frequency statistic analysis, the abstract sentence has diversity, and redundant information in the abstract can be effectively reduced. The system provides a simple and efficient automatic abstract extraction tool based on the method.
Description
technical field [0001] The invention relates to the field of natural language processing, in particular to an automatic abstract extraction method and system based on latent semantic analysis. Background technique [0002] With the rapid development of the Internet, the Internet has become the main channel for people to obtain information, and the content of document data on the Internet is also showing an exponential growth trend. Document data on the Internet contains a wealth of information, how to effectively read and filter useful information has become the focus of our attention. Automatic document summary technology compresses and expresses document information to help users better browse and absorb massive information on the Internet. [0003] Automatic summarization technology is a research hotspot in the field of natural language processing. According to the production method of the summary content, it can be divided into extractive summary and generative summary...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F17/30G06F17/27
CPCG06F16/345G06F40/289G06F40/30
Inventor罗强刘世林丁国栋
Owner成都数联铭品科技有限公司