ICLextRank dense point word extraction method based on fused semantics

Abstract

The invention relates to an ICLextRank dense point word extraction method based on fused semantics. The method comprises the following steps of text preprocessing, word graph construction, multi-feature fusion calculation and dense point word generation. The method comprises the following steps: firstly, performing sentence division, word segmentation, stop word removal and part-of-speech screening on an original document; secondly, constructing a word graph model based on a word co-occurrence relationship; then, the word features, the center features and the semantic features of the candidate words are synthesized for scoring, and text semantic vectors are extracted from the semantic features through a DistilBERT model and calculated through cosine similarity; and finally, sorting the candidate words according to the comprehensive score and outputting Top-K dense point words. According to the method, through combination of semantic fusion and centrality analysis, the accuracy and the intelligent level of dense point extraction are improved, and automatic auxiliary support is provided for electronic document secret setting.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to an ICLTextRank method for extracting key words based on fused semantics. Background Technology

[0002] With the widespread application of information technology and intelligent office systems, the number of electronic documents generated by government agencies and organizations has surged. Traditional manual classification suffers from low efficiency, strong subjectivity, and weak semantic recognition capabilities, making it difficult to meet the requirements of large-scale and consistent implementation. The revised Secrecy Law of 2024 requires government agencies and organizations to conduct classified information annotation in documents. However, existing automatic classification methods are mostly based on the traditional TextRank keyword extraction algorithm, which relies solely on word frequency and co-occurrence relationships for calculation, lacking semantic understanding and a public classified information database and automated annotation mechanism. This results in a prominent contradiction in classification practice: insufficient reusable knowledge, heavy manual burden, and low automation. In the scenario of classified information word extraction, this statistical algorithm often fails to identify classified words with implicit semantics, lacking consideration of semantic information, leading to low recognition accuracy. Furthermore, existing methods have a single feature dimension, failing to comprehensively consider the structural position, centrality, and semantic information of words, resulting in poor model interpretability and low feature utilization. Therefore, it is necessary to propose an improved algorithm that can integrate semantic and structural features to achieve high-precision identification of classified information. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide an ICLTextRank dense word extraction method based on semantic fusion. This method introduces three types of information—word features, center features, and semantic features—into the traditional TextRank algorithm. By fusing multi-dimensional feature information, it achieves high-precision automatic extraction of dense words from electronic documents.

[0004] Specifically, this method introduces a semantic embedding model (DistilBERT) to calculate the semantic similarity between candidate words and the confidentiality rules, and analyzes the importance of words in the word graph through a centrality algorithm. Finally, it fuses multi-dimensional features to obtain a comprehensive score, achieving efficient identification of potential confidentiality words. This method not only improves the algorithm's ability to identify semantically implicit confidentiality points, but also balances the interpretability and scalability of the algorithm while maintaining the flexibility of unsupervised feature extraction, making it suitable for confidentiality determination applications of various types of electronic documents.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for extracting dense words based on fused semantics using ICLTextRank is characterized by the following steps: Step 1, process the electronic document 分句, word segmentation, stop word removal, and part-of-speech filtering are performed in sequence to obtain a candidate word set ; where represents a word set, represents the first candidate word, represents the number of candidate words; The "stop words" mentioned above refer to words or characters that frequently appear in the text but have little semantic contribution, such as common function words like "了", "的", "对于", etc. Filtering out high-frequency but meaningless stop words helps reduce the feature dimension and mitigate the impact of noise. In this invention, a comprehensive set that combines Chinese, Harbin Institute of Technology, and Baidu stop word lists is selected.

[0006] Step 2: Using the candidate words obtained in Step 1 as nodes and the word collinearity relationship as edges, construct a word graph model ; where represents an edge set composed of word connection relationships, , and the subscript is only the number of each edge in the edge set; Step 3: Extract the word feature weights , central features and semantic features of each candidate word, and perform weighted fusion on these three features; Calculating the part-of-speech features of candidate words can more specifically increase or decrease the weights of different parts of speech, enabling the algorithm to focus on more important parts of speech. Through experimental analysis of a large amount of text corpora, more appropriate part-of-speech coefficients can be fitted. In the text pre-training stage, by performing part-of-speech tagging on various texts and screening out invalid word items, only nouns, verbs, and adjectives are retained as candidate keywords.

[0007] Step 4: Calculate the final score of candidate word , and output the top K words with high scores as dense words.

[0008] Based on the above solution, the steps for obtaining in Step 2 are as follows: Assume the sliding window size is , if the order of candidate word in a sentence is denoted as , and the position of candidate word in the sentence is index( ), when the position difference between index( ) and index( ) does not exceed , then there is a connecting edge , and each edge ​Number them to form an edge set ; Based on the above scheme, the steps described in step 3 are as follows: The calculation formula is shown below: ; in, Candidate words Word length characteristics; Candidate words In the document Word frequency features appearing in; Candidate words Part-of-speech features Candidate words Inverse document frequency features, Candidate words Intra-class distribution characteristics; Specifically: ; in, It is a candidate word Length, represent The maximum length of candidate words in the middle, The feature value representing the length of a word; ; in, It is a candidate word In the document Frequency of occurrence in; ; in, For the pos function; ; in, Represents the total number of documents in the corpus. Refers to the word containing The number of documents, Representative candidate words The inverse document frequency eigenvalue; ; in, Candidate words In the above Frequency distribution in the document For the above Documents in this document Candidate words Total number of occurrences; Based on the above scheme, the steps described in step 3 are as follows: The calculation formula is shown below: ; in, Candidate words in a word graph model and candidate words The shortest path distance between them Table Node The sum of the shortest path distances to other nodes in the word graph model.

[0009] Based on the above scheme, the steps described in step 3 are as follows: The calculation formula is shown below: ; in, To find the maximum value function, For each candidate word With sentences Cosine similarity, specifically: ; in, and Let the magnitude of the vector be . This represents the dot product of the candidate word vector and the sentence vector; For the detailed rules of confidentiality, for A sentence from the set of sentences obtained after sentence segmentation.

[0010] The ICLTextRank dense word extraction method based on fusion semantics described in this invention has the following beneficial effects: (1) Integrating multi-source features to achieve multi-dimensional modeling: This invention constructs an ICLTextRank dense word extraction method that integrates semantics. By introducing three types of information, namely word features, center features and semantic features, it realizes multi-dimensional representation and scoring of dense words in documents, effectively solving the limitation of the traditional TextRank algorithm that only relies on co-occurrence relationship and is difficult to identify implicit semantics.

[0011] (2) Constructing a word graph model with integrated features: Based on the unsupervised characteristics of TextRank, this invention combines multi-dimensional word features such as word length, part of speech, word frequency, inverse document frequency and intra-class distribution, and uses the Closeness Centrality algorithm to measure the centrality of candidate words in the word graph structure. Through network topology analysis, the algorithm’s ability to perceive global semantic associations is enhanced, making the evaluation of word importance more objective and hierarchical.

[0012] (3) Introducing a deep semantic modeling mechanism: By introducing the lightweight deep semantic model DistilBERT, this invention can extract the semantic vector representation between candidate words and the fixed density rules, calculate the cosine similarity between the two to obtain the semantic feature score, thereby enabling the algorithm to have semantic understanding and generalization ability, and significantly improving the recognition accuracy of implicit density information.

[0013] (4) Establish a comprehensive scoring system for feature fusion: This invention proposes a scoring mechanism for feature weighted fusion, which integrates word features, central features and semantic features to form a multi-dimensional dense point judgment index, realizing robust dense point extraction across document types and fields, and has high stability, portability and universality.

[0014] (5) It has intelligence and scalability: This method can be integrated with existing classification rule bases, semantic knowledge graphs and confidentiality management systems to form an automated and interpretable confidentiality annotation process, providing intelligent auxiliary classification support for government agencies, enterprises and research institutions, and has broad application prospects in the fields of confidentiality technology and natural language processing. Attached Figure Description

[0015] The present invention includes the following figures: Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the ICLTextRank word graph model; Figure 3 Flowchart for ICLTextRank word feature definition and calculation; Figure 4 Flowchart for defining and calculating the center features of ICLTextRank; Figure 5 Flowchart for defining and implementing semantic features of ICLTextRank; Figure 6 Visualize word clouds of the means of different algorithms across three datasets; Figure 7 Comparative experiments of different algorithms on the Fudan Set; Figure 8 A comparative experiment of different algorithms on THUCNews; Figure 9 This is a comparative experiment of different algorithms on a simulated dataset. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings.

[0017] like Figure 1As shown, the ICLTextRank dense word extraction method of the present invention includes the following steps: The original document is segmented into sentences, word-segmented, and filtered for stop words and parts of speech; a weighted word graph is built based on co-occurrence relationships, using candidate words as nodes. Extract word features, center features, and semantic features, and then weight and fuse the scores of the three features to calculate a comprehensive score. Sort the words in reverse order based on their overall scores, and output the top K high-scoring words as a set of dense words.

[0018] First, the original text is divided into sentences, and the resulting set of sentences is as follows: ; in, Represents a set of sentences. The index represents the first sentence in the set of sentences. This represents the total number of sentences. Then, the Jieba word segmentation tool is used to segment each sentence in the sentence set, extracting the corresponding word set. Next, stop words are removed to reduce feature dimensionality and mitigate noise, and part-of-speech tagging and filtering are performed. The candidate word set obtained after tagging and filtering the word set is as follows: ; in, Represents a set of words, Indicates the first candidate word. This indicates the number of candidate words. A word graph model is then constructed based on this, which will be described in detail below. Following this, the semantic feature extraction section optimizes the traditional TextRank algorithm in three aspects: It proposes the ICLTextRank algorithm based on fused semantics, defining feature attributes such as word length, word frequency, part-of-speech tagging, intra-class distribution, and inverse document frequency, and proposing methods for calculating these attributes to comprehensively extract the word characteristics of candidate words; it designs a method for calculating the center features of candidate words based on the Closeness Centrality concept, using a breadth-first search algorithm to expand the calculation of center features; and it designs a method that combines the DistilBERT model with domain-specific rules to calculate the semantic features of candidate words.

[0019] like Figure 2 As shown, the use of graph ranking algorithms requires the pre-construction of a word graph model. Typically, the word graph model is constructed using words as nodes, and connections between nodes are built using the collinear relationships between different words. By dividing the text into multiple word nodes, a word graph model is constructed using these nodes. The weight values ​​of the nodes are iteratively updated, and finally, the nodes are ranked according to their weights. Keywords are then extracted based on the ranking results.

[0020] After preprocessing the text data The set serves as the node set for the word graph model. Based on the co-occurrence relationships of words within a sliding window, the word graph model G=(W,E) is constructed, where W represents the set of all word nodes, and E represents the edge set formed by the connections between words. Assuming the sliding window size is If the term The order in a sentence is denoted as words The position in the sentence is index( When the positional difference between them does not exceed When, then there exists a connecting edge. , will each side Number them to form an edge set .

[0021] like Figure 3 As shown, the overall process includes: (1) Analysis of candidate word length features: First, we analyze the word length features of candidate words. Word length refers to the length of a word, which is one of the important features of text analysis. Longer words often carry more semantic information. The traditional TeatRank algorithm ignores the semantic features of words. By using a standardized method to calculate word length features, we can effectively increase the weight of long words in the algorithm, reduce the interference of short words, and further improve the accuracy of dense word extraction. The word length calculation formula is as follows: ; in, It is a candidate word Length, represent The maximum length of candidate words in the middle, The feature value of the representative word length.

[0022] (2) Frequency Feature Analysis of Candidate Words: Term frequency reflects the frequency of a word's appearance in a specific document. Frequently appearing words in a text are usually the core content. By using a smoothing method and a logarithmic function to calculate term frequency, we can avoid excessively high weight values ​​for high-frequency words, which could lead to uneven weight distribution. The term frequency calculation formula is as follows: ; in, It is a candidate word In the document Frequency of occurrence.

[0023] (3) Part-of-speech feature analysis of candidate words. Different parts of speech often carry different semantic relationships. Nouns represent different entities or core attributes, while verbs represent different states. Calculating the part-of-speech features of candidate words can more effectively increase or decrease the weight of different parts of speech, allowing the algorithm to focus on more important parts of speech. Through experimental analysis of a large amount of text corpus, more suitable part-of-speech coefficients can be fitted. In the text pre-training stage, by tagging various types of texts with parts of speech and filtering out invalid terms, only nouns, verbs, and adjectives are retained as candidate keywords. The pos function can calculate the weight coefficients corresponding to different parts of speech. The calculation formula is as follows: ; (4) Inverse Document Frequency Feature Analysis of Candidate Words: Inverse document frequency features are defined to reduce the weight of general words and highlight words with strong document discrimination capabilities. By introducing inverse document frequency features, the algorithm's discrimination ability can be enhanced. The calculation formula is as follows: ; in, Represents the total number of documents in the corpus. This refers to containing candidate words. The number of documents, Representative candidate words The inverse document frequency eigenvalue.

[0024] (5) Analysis of intra-class distribution features of candidate words: Intra-class distribution refers to the distribution of candidate words in different documents within the same category. Since the TextRank algorithm primarily relies on graph structures to calculate word importance, it neglects the different distributions and semantic situations of words within the same category. Therefore, by calculating intra-class distribution features, we can better capture the distribution differences of words and improve the recognition ability of candidate words. The calculation formula is as follows: ; in, The description is words Frequency distribution in the document set The calculation is Chinese words The total number of occurrences. In the formula above, if a word is more evenly distributed across different documents, the overall value of the formula will be small, because even distribution means that the word in each document is closer to the average number of times the word appears in the document, indicating that the word varies less across the document set.

[0025] (6) Word Feature Analysis of Candidate Words: In the algorithm, the weight of candidate words mainly depends on word collinearity, ignoring features such as word frequency, word length, and part-of-speech. Therefore, combining the word length, word frequency, part-of-speech, inverse document frequency, and intra-class distribution features proposed above, the improved word feature weights are calculated as follows: ; The formula for calculating the value of each node in the constructed word graph model is as follows: ; in, This indicates candidate words. Importance weights, It is the damping factor (usually set to 0.85). Is with Adjacent candidate words The weight. Then it means Transfer to The probability of the candidate word weights. Since the algorithm needs to iteratively calculate the weights of the candidate words, the convergence threshold is usually set to 0.001. When the calculation error of the node is less than 0.001 or the number of iterations reaches 100 rounds, the algorithm will stop iterating.

[0026] like Figure 4 As shown, considering that the traditional TextRank algorithm heavily relies on the collinearity of candidate words, it may overlook words that appear less frequently but are representative and core. Therefore, this section introduces the ClosenessCentrality algorithm, which calculates the shortest path between each candidate word and other candidate words, identifying words closely related to other candidate words in the entire document. By calculating the central features of candidate words, it is possible to extract words that appear less frequently but are semantically important, which helps improve the accuracy of candidate word extraction. This includes: (1) Closeness Centrality subgraph: Assume there exists an undirected graph ,in, Representing the set of nodes in a word, the Closeness Centrality algorithm considers nodes... Centrality is defined as: ; in, The central feature score represents the node. Represents a node and nodes The shortest path distance between them Represents a node The sum of the shortest path distances to other nodes in the word graph model.

[0027] (2) Closeness Centrality Algorithm Flow: First, input the word graph model constructed in the previous steps. It initializes a Closeness Centrality set to store the central feature value of each node, defines a distance matrix to store the shortest paths between different candidate words, and then uses a breadth-first search algorithm to calculate the shortest path distance between different nodes, while updating the distance matrix. It iterates through all nodes, and for each node... For each candidate node, a center feature score is calculated, and the feature value is stored in the Closeness Centrality set. The higher the center feature score, the closer the node is to the center position, and the stronger its representativeness of the text. Finally, word features and center feature scores are fused for dense point extraction.

[0028] like Figure 5 As shown, considering the practical characteristics of dense point extraction, based on the calculation of word features and semantic features of candidate words, the specific dense point rules are incorporated into the dense point extraction process. The semantic features of candidate words are calculated, and the DistilBERT model is used to calculate the sentence vector of the dense point rules. Simultaneously, word vectors from the candidate word set are generated. Cosine similarity is used to calculate the semantic similarity between each candidate word and the specific dense point rules to obtain the semantic feature score. The DistilBERT model is a compressed version of the BERT model, which can quickly process large amounts of text data while retaining the performance of the BERT model. Due to its internal Transformer architecture, the DistilBERT model can capture semantic information in the text, improving the accuracy of semantic feature calculation. The specific calculation steps are as follows: Assumption Given a set of sentences in the confidentiality rules, the rules are segmented into sentences. The resulting set of sentences is represented as follows: For each Use the DistilBERT model to generate sentence vectors For candidate word set In Generate word vectors For each candidate word Sentences in the Calculation and Confidentiality Rules Cosine similarity:

[0029] in, and Let be the magnitude of the vector (i.e., the Euclidean norm). This represents the dot product of the candidate word vector and the sentence vector. For candidate words... The highest cosine similarity is selected from the specified rules as the semantic feature score: ; Generate a set of semantic feature scores for each candidate word: ; The ICLTextRank algorithm calculates the final score of a candidate word w, which consists of three parts: word features, center features, and semantic features. The final score calculation formula is: ; Based on the original TextRank algorithm, improvements are made in five dimensions: word length, word frequency, part-of-speech tagging, inverse document frequency, and intra-class distribution, to calculate the word features of candidate words. The Closeness Centrality algorithm combined with breadth-first search can be used to calculate the central features of candidate words on the word graph model. The semantic features of candidate words can be obtained by calculating the semantic similarity between candidate words and the density rules. The candidate words are sorted in reverse order by combining the scores of word features, central features, and semantic features, and the top K words with the highest scores are output as density words.

[0030] The semantic feature extraction described in this invention can also be performed using other pre-trained language models and vector semantic models. These models include BERT, RoBERTa, ERNIE, ELECTRA, or SimCSE, all of which can calculate the semantic similarity between candidate words and the defined rules, thereby obtaining semantic feature scores. Using lightweight models (such as MiniLM) can further improve processing speed and deployment efficiency.

[0031] Example 1: Core Comparison Effect Experiment To verify the effectiveness of the proposed ICLTextRank dense word extraction method based on fused semantics, this invention selects several common keyword / dense word extraction algorithms as comparative baselines, including CHI, LDA, TextRank, and VTextRank. To further illustrate the advantages of the algorithm in dense word extraction tasks, multiple datasets are selected to test the performance of the above algorithms, with precision (P), recall (R), and F1 score used as evaluation metrics. The experimental statistical results are shown in Table 1.

[0032] Table 1 Comparison of Keyword Extraction Algorithms ;

[0033] As can be seen, the improved ICLTextRank algorithm achieves an average F1 score of 81.3, which is better than other compared algorithms and an improvement over TextRank's 76.6. This is because TextRank relies only on local contextual information, while the proposed ICLTextRank algorithm comprehensively considers information from multiple dimensions, including word features, semantic features, and central features. Word cloud visualization provides a more intuitive view and understanding of the text's main content. Figure 6 As can be seen, the algorithm proposed in this invention can accurately and efficiently display text data information when extracting keywords, thus proving the effectiveness of the method.

[0034] Example 2: Method Application Experiment To verify the effectiveness of the proposed ICLTextRank dense word extraction method in density-level classification tasks, this embodiment uses the dense words extracted by this method as feature input and compares them with several common machine learning classification models. To ensure the fairness of the comparison, each model uses Word2Vec vectors as text representation.

[0035] Table 2 Comparison of Keyword Extraction Algorithms ;

[0036] The results show that the ICLTextRank method proposed in this invention achieves the best classification performance on all three datasets, with F1 scores of 84.3, 81.2, and 78.5, respectively, significantly higher than traditional machine learning models such as DT, KNN, and LR. While the SVM model showed relatively stable overall performance in the comparison algorithms, it still lagged behind the extraction results of this invention, indicating that the dense point words in this invention have higher confidence in expressing density-level related information.

[0037] The overall classification performance of the simulated dataset is lower than that of the news dataset, mainly due to the smaller number of simulated samples and the imbalanced class distribution, making it difficult for each model to fully learn its features. To more intuitively demonstrate the performance of each model on different datasets, this example presents a bar chart comparing classification precision (P), recall (R), and F1 score, as shown below. Figure 7 , Figure 8 , Figure 9 As shown in the figure, it is clear from the figure that the ICLTextRank method of this invention performs better on multiple datasets.

[0038] This invention pertains to electronic document confidentiality determination assistance technology within the field of network security. While ensuring the accuracy of semantic and structural feature extraction, it significantly improves the interpretability, rationality, and intelligence of the confidentiality identification process. By integrating word features, center features, and semantic features, this invention can fully integrate multi-source information reflecting confidentiality features in documents, taking into account both the semantic constraints of confidentiality rules and the statistical characteristics of document structure. This results in more accurate, comprehensive, and semantically consistent confidentiality extraction results, effectively supporting the automation and decision-making assistance in electronic document confidentiality determination.

[0039] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A method for extracting dense words based on fused semantics using ICLTextRank, characterized in that, Includes the following steps: Step 1, process the electronic document The process involves sentence segmentation, word segmentation, stop word removal, and part-of-speech tagging to obtain a candidate word set. ;in, Represents a set of words, Indicates the first candidate word. Indicates the number of candidate words; Step 2: Using the candidate words obtained in Step 1 as nodes and the collinear relationships between words as edges, construct a word graph model. ;in An edge set representing the connections between words. Subscript Only edge sets China and its allies The number; Step 3: Extract the word feature weights of each candidate word. Central features and semantic features We then perform a weighted fusion of these three features; Step 4, Calculate candidate words Final score The top K words with the highest scores are output as the dense words.

2. The method for extracting dense words based on fused semantics according to claim 1, characterized in that: The edge set mentioned in step 2 The steps to obtain it are as follows: Assuming the sliding window size is If candidate words The order in a sentence is denoted as Candidate words The position in the sentence is ,when and The positional difference between them does not exceed When, then there exists a connecting edge. , will each side Number them to form an edge set .

3. The method for extracting dense words based on fused semantics according to claim 1, characterized in that: The steps described in step 3 The calculation formula is shown below: ; in, Candidate words Word length characteristics; Candidate words In the document Word frequency features appearing in; Candidate words Part-of-speech features Candidate words Inverse document frequency features, Candidate words Intra-class distribution characteristics; Specifically: ; in, It is a candidate word Length, represent The maximum length of candidate words in the middle, The feature value representing the length of a word; ; in, It is a candidate word In the document Frequency of occurrence in; ; in, For the pos function; ; in, Represents the total number of documents in the corpus. Refers to the word containing The number of documents, Representative candidate words The inverse document frequency eigenvalue; ; in, Candidate words In the above Frequency distribution in the document For the above Documents in this document Candidate words Total number of occurrences.

4. The method for extracting dense words based on fused semantics according to claim 1, characterized in that: The steps described in step 3 The calculation formula is shown below: ; in, Candidate words in a word graph model and candidate words The shortest path distance between them Represents a node The sum of the shortest path distances to other nodes in the word graph model.

5. The method for extracting dense words based on fused semantics in ICLTextRank as described in claim 1, characterized in that: The steps described in step 3 The calculation formula is shown below: ; in, To find the maximum value function, For each candidate word With sentences Cosine similarity, specifically: ; in and Let the magnitude of the vector be . This represents the dot product of the candidate word vector and the sentence vector; For the detailed rules of confidentiality, for A sentence from the set of sentences obtained after sentence segmentation.

Images