Unsupervised extraction type text summarization method

Through the unsupervised extracted text summary method, high-dimensional word table and sentence vector representation, iterative computing and cluster division technology, the existing extracted digest performance is solved, and the generation and calculation efficiency of high-quality digests are improved. .

CN118964604APending Publication Date: 2024-11-15ITIBIA TECH (SUZHOU) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411030676.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing extracted text summary method has shortcomings in performance improvement, especially in scenarios where high performance is required. The traditional method relies on a large number of labeled samples, resulting in high training costs and poor universality.

Method used

The unsupervised abstract text summary method is used to preprocess the document, generate high-dimensional word tables and sentence vectors, and use iterative computing and cluster division techniques to extract the core sentences of the document to generate high-quality abstracts, reducing the calculation cost.

Benefits of technology

In the scenario where the digest labeled data is missing, higher quality summary content acquisition is achieved, information coverage, loyalty and computing speed are improved, and performance benchmarks are exceeded by traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964604A_ABST
    Figure CN118964604A_ABST
Patent Text Reader

Abstract

The invention discloses an unsupervised extraction type text abstract method, which comprises the following steps of: preprocessing a document, generating high-dimensional vectors for characters appearing in the document, calculating vectors of sentences according to the character vectors, and calculating to obtain core sentences in the document to obtain a final abstract. Designing a sentence vector representation form more suitable for the text abstract by utilizing the pseudo orthogonality of a high-dimensional vector; the core sentences of the document can be more reasonably found through iterative calculation, and the comparison benchmark of the method of the same type in the aspects of information coverage, loyalty, calculation speed and the like is surpassed; the abstract content with higher quality can be obtained at lower calculation cost in a scene lacking abstract annotation data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an unsupervised extraction text summarization method, and belongs to the technical field of natural language processing. Background Art

[0002] Text summarization is an important branch of natural language processing, which aims to extract key information from a large amount of text and generate concise and coherent summaries. With the explosive growth of Internet information, people's demand for rapid access to information has become increasingly urgent, and text summarization technology has therefore developed rapidly. Text summarization technology has mainly experienced a transition from rule-based methods to statistical methods, and then to modern machine learning-based methods.

[0003] Text summarization can be divided into extractive summarization and generative summarization according to the output type. Extractive summarization extracts key sentences and keywords from the source document to form a summary, and the summary is entirely derived from the original text. Generative summarization allows the generation of new words and phrases to form a summary based on the original text. Both methods have their own advantages and disadvantages. With the widespread use of deep neural networks, generative summarization using deep learning methods such as sequence-to-sequence has been studied more, while research on extractive summarization has stagnated somewhat.

[0004] From the perspective of statistical learning methods, text summarization includes unsupervised methods and supervised learning methods. Traditional text summarization methods usually require a large number of labeled summary samples for training before generating summaries. However, labeling samples requires a lot of manpower and time costs, and the quality of labeling may be affected by subjective factors. Unsupervised methods do not rely on labeled samples, but use the characteristics and structure of the text itself to generate summaries, thereby reducing training costs and improving universality and scalability.

[0005] At present, extractive text summarization methods include methods based on word frequency statistics, methods based on word frequency-inverse document frequency (TF-IDF), methods based on graph ranking (such as TextRank algorithm), etc. Although they are practical in some scenarios with low requirements, their performance needs to be improved in many cases. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an unsupervised extractive text summarization method.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] The unsupervised extractive text summarization method is characterized by: comprising the following steps:

[0009] S1) preprocessing the document;

[0010] S2) Generate high-dimensional vectors for words appearing in the document;

[0011] S21) Counting the words used in the document to establish a word table;

[0012] Based on the preprocessed sentence list, a word list of the document is generated using statistical methods. Words that appear multiple times in the document are only included once when they first appear. High-frequency words appear multiple times in the document, and the length of the final word list is much smaller than the number of words in the original document.

[0013] S22) initializing a vector for each word in the word table;

[0014] To convert each word into a vector representation in a high-dimensional space, first set the dimension D of the vector;

[0015] The vector Ci of the i-th word in the word list is obtained by formula (1) and formula (2);

[0016] Ci = [T(i,0) ,T(i,1) ,T(i,2) ……,T(i,D-1) ] (1)

[0017]

[0018] Where i is the index of the word in the word table, and d is the ordinal number of the vector dimension, both starting from 0; T(i,d) represents the value of the i-th word in the d-th dimension;

[0019] S3) calculating each sentence vector based on the word vector;

[0020] Extractive summarization is to select sentences from the original document. In order to characterize the importance of sentences to the overall semantics of the document, after obtaining the vector of each word in the word list in step S2), a sentence vector is generated for each sentence based on the word vector.

[0021] For a given sentence S', the sentence vector Vec(S') is obtained by using formula (3) for each word vector;

[0022]

[0023] Where Sum is a vector summation operation, n is the number of valid words in the sentence S', j is the index of the current word C in the sentence, and the Shift(Cj, nj-1) function means to shift the vector Cj right nj-1 times, and move the nj-1 elements at the end of the vector to the front;

[0024] After right shifting each word vector for different times, the sum is calculated to get the vector representation of the sentence. The dimension of the vector remains unchanged during the calculation process, and the dimension of the sentence vector is consistent with that of the word vector.

[0025] For the sentence list obtained in step S1), after step S2) and step S3), a D-dimensional vector representation of each sentence in the list is obtained;

[0026] S4) calculating and obtaining the core sentences in the document;

[0027] In the document summarization scenario, the most important sentences in the document are extracted as the central idea or the gist of the article; the number of sentences k required for the final summary is regarded as a known parameter in advance and provided each time the summary task is performed; after the vectors of each sentence are obtained in step S3), the relationship between the sentences is calculated to obtain the most core k sentences in the document;

[0028] S5) obtain the final summary;

[0029] After obtaining the sentence vectors of k core sentences in step S4), they are sorted according to the index numbers in the original sentence list, and then the original sentences of the document before preprocessing corresponding to each index are found, and they are combined in order to obtain the final summary of the document.

[0030] Furthermore, the above unsupervised extractive text summarization method, wherein:

[0031] Step S1), single document summary, a document consists of multiple sentences, the number of sentences required for the target summary is less than the number of sentences contained in the original document, in order to identify more core sentences, the document is preprocessed; including:

[0032] S11) Segmenting the document;

[0033] Use punctuation marks to divide the document into several sentences, and then form a sentence list in the form of [S1, S2, S3…, Sn];

[0034] S12) preprocessing each sentence;

[0035] In order to facilitate the statistics of valid words in the document, the sentences are preprocessed. For Chinese documents, the preprocessing includes removing stop words, punctuation marks, special symbols, foreign language vocabulary and non-Chinese characters, leaving only Chinese characters and digital texts;

[0036] After preprocessing, the original sentence list [S1, S2, S3…, Sn] becomes [S'1, S'2, S'3…, S'n].

[0037] Furthermore, in the above-mentioned unsupervised extractive text summarization method, in step S22), the dimension D is 5000, 10000 or 20000.

[0038] Furthermore, the above unsupervised extractive text summarization method, wherein:

[0039] Step S4) comprises the following:

[0040] S41) randomly selecting k vectors from the sentence vector list as cores;

[0041] S42) calculating the distances from the remaining vectors in the list to the core vectors;

[0042] S43) dividing the sentence vectors outside the core into the core with the closest distance, and dividing the sentence vector list into k clusters;

[0043] S44) recalculating the core in each cluster according to the divided k clusters; firstly calculating the distance from all vectors in the cluster to one of the vectors, which is called the absolute error; then selecting the vector that minimizes the absolute error in the cluster as the new core of the cluster;

[0044] S45) Iterate step S42) to step S44) until the k cores no longer change, or stop when a preset maximum number of iterations is reached.

[0045] Furthermore, in the above-mentioned unsupervised extractive text summarization method, in step S42), the distance measurement method of the vector includes Euclidean distance, Manhattan distance, and cosine distance.

[0046] Furthermore, in the above-mentioned unsupervised extractive text summarization method, in step S43), if a certain vector v is at an equal distance from several target clusters; if the target cluster contains the cluster to which v belongs, the affiliation of v remains unchanged; if the target cluster does not contain the cluster to which v belongs, a target cluster is randomly selected and divided.

[0047] Compared with the prior art, the present invention has significant advantages and beneficial effects, which are specifically embodied in the following aspects:

[0048] The present invention utilizes the pseudo-orthogonality of high-dimensional vectors to design a sentence vector representation form that is more suitable for text summarization; it is more reasonable to find the core sentence of the document through iterative calculation, and it surpasses the previous benchmarks of similar methods in terms of information coverage, fidelity, and calculation speed; the method of the present invention can obtain higher quality summary content at a lower calculation cost in scenarios where summary annotation data is lacking.

[0049] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the specific embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 : Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0053] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, directional terms and order terms are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0054] like Figure 1 As shown, the unsupervised extractive text summarization method includes the following steps:

[0055] S1) preprocessing the document;

[0056] For single-document summarization, a document consists of multiple sentences. The number of sentences required for the target summary is less than the number of sentences contained in the original document. In order to identify more core sentences, the document needs to be preprocessed; including:

[0057] S11) Segmenting the document;

[0058] Use punctuation marks to divide the document into several sentences, and then form a sentence list, such as [S1, S2, S3..., Sn];

[0059] S12) preprocessing each sentence;

[0060] In order to facilitate the statistics of valid words in the document, the sentences are preprocessed. For Chinese documents, the preprocessing includes removing stop words, punctuation marks, special symbols, foreign language vocabulary and non-Chinese characters, leaving only Chinese characters and digital texts;

[0061] After preprocessing, the original sentence list [S1, S2, S3…, Sn] becomes [S'1, S'2, S'3…, S'n];

[0062] S2) Generate high-dimensional vectors for words appearing in the document;

[0063] S21) Counting the words used in the document to establish a word table;

[0064] Based on the preprocessed sentence list, a word list of the document is generated using statistical methods. Words that appear multiple times in the document are only included once when they first appear. High-frequency words appear multiple times in the document, and the length of the final word list is much smaller than the number of words in the original document.

[0065] S22) initializing a vector for each word in the word table;

[0066] In order to convert each word into a vector representation in a high-dimensional space, the dimension D of the vector is first set. Considering that the coverage of common words in different languages ​​varies, the dimension D is selected to be 5000, 10000, 20000 or higher; specifically, D=5000 is selected for the Chinese corpus;

[0067] The vector Ci of the i-th word in the word list is obtained by formula (1) and formula (2);

[0068] Ci = [T(i,0) ,T(i,1) ,T(i,2) ……,T(i,D-1) ] (1)

[0069]

[0070] Where i is the index of the word in the word table, and d is the ordinal number of the vector dimension, both starting from 0; T(i,d) represents the value of the i-th word in the d-th dimension;

[0071] The 0th word vector in the word table is [1,1,1,1,1…1], the 1st word vector is [-1,1,1,1,1…1], the 2nd word vector is [-1,-1,1,1,1…1], and so on;

[0072] S3) calculating each sentence vector based on the word vector;

[0073] Extractive summarization is to select sentences from the original document. In order to characterize the importance of sentences to the overall semantics of the document, after obtaining the vector of each word in the word list in step S2), a sentence vector is generated for each sentence based on the word vector.

[0074] For a given sentence S', the sentence vector Vec(S') is obtained by using formula (3) for each word vector;

[0075]

[0076] Where Sum is a vector summation operation, n is the number of valid words in the sentence S', j is the index number of the current word C in the sentence (starting from 0), and the Shift(Cj, nj-1) function means to shift the vector Cj right nj-1 times, and move the nj-1 elements at the end of the vector to the front;

[0077] A vector [-1,-1,1,1,1,1,1,1,1,1], if shifted right once, the result is [1,-1,-1,1,1,1,1,1,1,1]; and if shifted right three times, the result is [1,1,1,-1,-1,1,1,1,1,1];

[0078] After right shifting each word vector for different times, the sum is calculated to get the vector representation of the sentence. The dimension of the vector remains unchanged during the calculation process, and the dimension of the sentence vector is consistent with that of the word vector.

[0079] For the sentence list obtained in step S1), after step S2) and step S3), a D-dimensional vector representation of each sentence in the list is obtained;

[0080] S4) calculating and obtaining the core sentences in the document;

[0081] In the document summarization scenario, the three to five most important sentences in the document are extracted as the central idea or the gist of the article; the number of sentences k required for the final summary is regarded as a known parameter in advance and provided each time the summary task is performed; after the vectors of each sentence are obtained in step S3), the relationship between the sentences is calculated to obtain the most core k sentences in the document; the steps are as follows:

[0082] S41) randomly selecting k vectors from the sentence vector list as cores;

[0083] The k value is determined according to the actual business needs; specifically, the k value is 5;

[0084] S42) calculating the distances from the remaining vectors in the list to the core vectors;

[0085] Vector distance metrics include Euclidean distance, Manhattan distance, and cosine distance;

[0086] S43) dividing the sentence vectors outside the core into the core with the closest distance, and dividing the sentence vector list into k clusters;

[0087] If a vector v is equidistant from several target clusters; if the target cluster contains the cluster to which v belongs, the affiliation of v remains unchanged; if the target cluster does not contain the cluster to which v belongs, a target cluster is randomly selected and divided into it;

[0088] S44) According to the divided k clusters, the core is recalculated in each cluster; first, the distance from all vectors in the cluster to one of the vectors is calculated, which is called the absolute error; then, the vector that minimizes the absolute error in the cluster is selected as the new core of the cluster;

[0089] S45) iteratively executing step S42) to step S44) until the k cores no longer change, or the preset maximum number of iterations is reached;

[0090] The maximum number of iterations can be set to 100;

[0091] S5) obtain the final summary;

[0092] After obtaining the sentence vectors of k core sentences in step S4), they are sorted according to the index numbers in the original sentence list, and then the original sentences of the document before preprocessing corresponding to each index are found, and they are combined in order to obtain the final summary of the document.

[0093] In summary, the present invention uses the pseudo-orthogonality of high-dimensional vectors to design a sentence vector representation form that is more suitable for text summarization; it is more reasonable to find the core sentence of the document through iterative calculation, and it surpasses the previous benchmarks of similar methods in terms of information coverage, fidelity, and calculation speed. The method of the present invention can obtain higher-quality summary content at a lower computational cost in scenarios where summary annotation data is lacking, and the present invention has a wide range of application value in unsupervised extractive summarization.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention. It should be noted that similar numbers and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0095] The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0096] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. Unsupervised extractive text summarization method, Features: The following steps are involved: S1) preprocessing the document; S2) Generate high-dimensional vectors for words appearing in the document; S21) Counting the words used in the document to establish a word table; Based on the preprocessed sentence list, a word list of the document is generated using statistical methods. Words that appear multiple times in the document are only included once when they appear for the first time. High-frequency words appear multiple times in a document, and the length of the final word list is much smaller than the number of words in the original document; S22) initializing a vector for each word in the word table; To convert each word into a vector representation in a high-dimensional space, first set the dimension D of the vector; The vector Ci of the i-th word in the word list is obtained by formula (1) and formula (2); Ci = [T(i,0),T(i,1),T(i,2)...,T(i,D-1)] (1) Where i is the index of the word in the word table, and d is the ordinal number of the vector dimension, both starting from 0; T(i,d) represents the value of the i-th word in the d-th dimension; S3) calculating each sentence vector based on the word vector; Extractive summarization is to select sentences from the original document. In order to characterize the importance of sentences to the overall semantics of the document, after obtaining the vector of each word in the word list in step S2), a sentence vector is generated for each sentence based on the word vector. For a given sentence S', the sentence vector Vec(S') is obtained by using formula (3) for each word vector; Where Sum is a vector summation operation, n is the number of valid words in the sentence S', j is the index of the current word C in the sentence, and the Shift(Cj, nj-1) function means to shift the vector Cj right nj-1 times, and move the nj-1 elements at the end of the vector to the front; After right shifting each word vector for different times, the sum is calculated to get the vector representation of the sentence. The dimension of the vector remains unchanged during the calculation process, and the dimension of the sentence vector is consistent with that of the word vector. For the sentence list obtained in step S1), after step S2) and step S3), a D-dimensional vector representation of each sentence in the list is obtained; S4) calculating and obtaining the core sentences in the document; In the document summarization scenario, the most important sentences in the document are extracted as the central idea or the gist of the article; the number of sentences k required for the final summary is regarded as a known parameter in advance and provided each time the summary task is performed; after the vectors of each sentence are obtained in step S3), the relationship between the sentences is calculated to obtain the most core k sentences in the document; S5) obtain the final summary; After obtaining the sentence vectors of k core sentences in step S4), they are sorted according to the index numbers in the original sentence list, and then the original sentences of the document before preprocessing corresponding to each index are found, and they are combined in order to obtain the final summary of the document.

2. The unsupervised extractive text summarization method according to claim 1, Features: Step S1), single document summary, a document consists of multiple sentences, the number of sentences required for the target summary is less than the number of sentences contained in the original document, in order to identify more core sentences, the document is preprocessed; include: S11) Segmenting the document; Use punctuation marks to divide the document into several sentences, and then form a sentence list in the form of [S1, S2, S3…, Sn]; S12) preprocessing each sentence; In order to facilitate the statistics of valid words in the document, the sentences are preprocessed. For Chinese documents, the preprocessing includes removing stop words, punctuation marks, special symbols, foreign language vocabulary and non-Chinese characters, leaving only Chinese characters and digital texts; After preprocessing, the original sentence list [S1, S2, S3…, Sn] becomes [S'1, S'2, S'3…, S'n].

3. The unsupervised extractive text summarization method according to claim 1, Features: Step S22), dimension D is 5000, 10000 or 20000.

4. The unsupervised extractive text summarization method according to claim 1, Features: Step S4) comprises the following: S41) randomly selecting k vectors from the sentence vector list as cores; S42) calculating the distances from the remaining vectors in the list to the core vectors; S43) dividing the sentence vectors outside the core into the core with the closest distance, and dividing the sentence vector list into k clusters; S44) recalculating the core in each cluster according to the divided k clusters; firstly calculating the distance from all vectors in the cluster to one of the vectors, which is called the absolute error; then selecting the vector that minimizes the absolute error in the cluster as the new core of the cluster; S45) Iterate step S42) to step S44) until the k cores no longer change, or stop when a preset maximum number of iterations is reached.

5. The unsupervised extractive text summarization method according to claim 4, Features: Step S42), the distance measurement methods of the vector include Euclidean distance, Manhattan distance, and cosine distance.

6. The unsupervised extractive text summarization method according to claim 4, Features: Step S43), if a vector v is at an equal distance from several target clusters; if the target cluster contains the cluster to which v belongs, the affiliation of v remains unchanged; if the target cluster does not contain the cluster to which v belongs, a target cluster is randomly selected and divided into.