Keyword generation method and device

Abstract

The invention discloses a keyword generation method and device, and relates to the technical field of keyword generation, and the keyword generation method comprises the following steps: carrying out word segmentation processing on a semantic unit corresponding to a document set to obtain a lexical item set, the semantic unit being obtained by splitting an electric power regulation document in the document set; based on the importance degree of each lexical item in the lexical item set to the document set, screening the lexical items to form a candidate lexical item set; performing semantic merging processing and topic aggregation processing on candidate lexical items in the candidate lexical item set in sequence to obtain a global keyword item set; based on the semantic similarity between the semantic units and the global keyword items, screening adaptive global keyword items for each semantic unit from the global keyword item set, and generating an exclusive keyword item set corresponding to each semantic unit; and based on the degree that the candidate lexical items surround high-frequency lexical items in the global keyword item set semantically, screening out long-tail lexical items from the candidate lexical items, and adding the long-tail lexical items into the exclusive keyword item set corresponding to each semantic unit.

Description

Technical Field

[0001] This application relates to the field of keyword generation technology, and in particular to a keyword generation method and apparatus. Background Technology

[0002] With the development of digitalization and intelligentization of power systems, the number of power regulations and documents is constantly increasing. These documents typically have a fixed hierarchy, standardized language, but complex semantics. To facilitate knowledge extraction and high-precision retrieval, documents are usually divided into several semantic units, and a unique set of keywords is generated for each unit to accurately represent its core semantics. This supports rapid location of regulations clauses, cross-document knowledge point association, and semantic matching by intelligent question-and-answer systems.

[0003] Currently, traditional keyword generation methods are mostly based on word frequency statistics (such as TF-IDF, TextRank, etc.) or semantic embedding (such as Word2Vec, BERT, etc.). However, these methods have the following drawbacks: First, word frequency statistics-based methods are easily affected by high-frequency template words, resulting in keywords that lack representativeness and are difficult to accurately summarize the semantic meaning of semantic units. Second, semantic embedding-based methods cannot effectively distinguish between content segments with similar structures but large semantic differences, leading to semantic matching deviations between keywords and semantic units. Third, low-frequency and business-critical terms in power regulations are often ignored, making it difficult for keywords to cover power industry business knowledge. Therefore, existing keyword generation methods are insufficient to generate keyword sets that conform to power industry standards and accurately summarize the semantic meaning of semantic units.

[0004] Therefore, how to generate a set of keywords that conforms to power industry standards and accurately summarizes the semantic theme for the semantic units of power regulations documents has become an urgent problem to be solved. Summary of the Invention

[0005] This application proposes a keyword generation method and apparatus, the main purpose of which is to generate a set of keywords that conform to power industry standards and accurately summarize the semantic theme for the semantic units of power regulations documents.

[0006] To achieve the above objectives, this application mainly provides the following technical solutions: Firstly, this application provides a keyword generation method. The keyword generation method provided in this embodiment may include at least: performing word segmentation on semantic units corresponding to a document set to obtain a term set, wherein the semantic units are obtained by splitting the power regulations documents in the document set; filtering terms based on the importance of each term in the term set to the document set to form a candidate term set; sequentially performing semantic merging and topic aggregation processing on the candidate terms in the candidate term set to obtain a global keyword term set; based on the semantic similarity between the semantic units and the global keyword terms, filtering suitable global keyword terms from the global keyword term set for each semantic unit to generate a dedicated keyword term set corresponding to each semantic unit; and filtering long-tail terms from the candidate terms based on the degree to which the candidate terms semantically surround the high-frequency terms in the global keyword term set, and adding them to the dedicated keyword term set corresponding to each semantic unit.

[0007] Secondly, this application provides a keyword generation device, which in this embodiment may include at least: The word segmentation module is used to perform word segmentation on the semantic units corresponding to the document set to obtain a term set. The semantic units are obtained by splitting the power regulations documents in the document set. The filtering module is used to filter terms based on the importance of each term in the term set to the document set, thereby forming a candidate term set; The processing module is used to sequentially perform semantic merging and topic aggregation processing on the candidate terms in the candidate term set to obtain a global keyword term set; The generation module is used to select suitable global keyword items for each semantic unit from the global keyword item set based on the semantic similarity between the semantic unit and the global keyword items, and generate a unique keyword item set for each semantic unit. An addition module is used to filter long-tail keywords from candidate keywords based on the degree to which candidate keywords semantically surround high-frequency keywords in the global keyword set, and add them to the dedicated keyword set corresponding to each semantic unit.

[0008] Thirdly, this application provides a computer-readable storage medium including a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the keyword generation method of the first aspect.

[0009] Fourthly, this application provides an electronic device, the electronic device comprising: a memory for storing a program; and a processor coupled to the memory for running the program to execute the keyword generation method of the first aspect.

[0010] Fifthly, this application provides a computer program product, the computer program product comprising: a computer program / computer executable instructions, and the computer program / computer executable instructions comprising the keyword generation method of the first aspect.

[0011] The keyword generation method and apparatus provided in this application, upon determining that keywords need to be generated, first obtains semantic units corresponding to a document set. These semantic units are obtained by splitting the power regulations documents within the document set. Then, the semantic units are segmented to obtain a term set. Next, terms are selected based on the importance of each term in the term set to the document set, forming a candidate term set. Then, the candidate terms in the candidate term set are sequentially subjected to semantic merging and topic aggregation processing to obtain a global keyword term set. Then, based on the semantic similarity between the semantic units and the global keyword terms, suitable global keyword terms are selected from the global keyword term set for each semantic unit, generating a dedicated keyword term set for each semantic unit. Finally, based on the degree to which the candidate terms semantically revolve around the high-frequency terms in the global keyword term set, long-tail terms are selected from the candidate terms and added to the dedicated keyword term set corresponding to each semantic unit. As can be seen, the solution provided in this embodiment first focuses keywords on the core content and key information of power regulations documents through term filtering, avoiding the generation of irrelevant or secondary keywords. Then, it integrates scattered terms with overlapping meanings through semantic merging, eliminating redundancy and internal competition between keywords, making the expression more concise and highlighting the core concepts. Subsequently, it mines and establishes semantic relationships between terms through semantic aggregation, constructing semantic thematic term clusters, thereby significantly expanding the breadth and depth of keyword coverage of the document's main theme. The targeted supplementation of long-tail keywords effectively captures low-frequency but business-critical terms in the regulations documents that have professional differentiation but are easily overlooked, significantly improving the uniqueness of the keyword set and the retrieval recall rate. Finally, the keyword set generated by the keyword generation solution provided in this embodiment not only strictly conforms to the terminology norms of the power industry, but also achieves high-precision semantic matching with the content of power regulations documents, thus providing effective support for subsequent retrieval, knowledge management, and content recommendation.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart of a keyword generation method provided in one embodiment of this application is shown; Figure 2 This illustration shows a schematic diagram of a keyword generation device according to an embodiment of this application; Figure 3 A schematic diagram of a keyword generation device provided in another embodiment of this application is shown. Detailed Implementation

[0015] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Currently, there are numerous power regulations documents in the power system, and these documents often share a highly consistent structure across different devices or scenarios, such as similar chapter titles and clause formats. However, the content they describe, such as the equipment objects and operational logic, varies significantly. This characteristic of "structural consistency but strong semantic isolation" makes it difficult for traditional keyword generation methods based on word frequency statistics or semantic embedding to generate keyword sets that conform to power industry standards and accurately summarize the core meaning of semantic units (i.e., the smallest text units that can independently express complete meaning) in power regulations. Consequently, keywords cannot distinguish the semantic differences between various semantic units across devices or scenarios, making it difficult to effectively support rapid location of regulation clauses, cross-document knowledge point association, and semantic matching in intelligent question-and-answer systems based on keywords.

[0016] Research has shown that after obtaining the semantic units corresponding to power regulations documents in the document set, the semantic units can be segmented to obtain a term set. Then, some terms are selected from this term set as candidate term sets. Next, semantic merging is used to eliminate redundancy and fragmentation in the candidate term set, ensuring the conciseness and consistency of core concepts. Then, semantic associations between candidate terms are identified, and topic aggregation is performed to obtain a global keyword set suitable for the document set. Next, based on the semantic similarity between the semantic units and the global keyword set, suitable global keyword items are selected from the global keyword set for each semantic unit, generating a dedicated keyword set for each semantic unit. Finally, fine-grained concepts that are easily overlooked but have professional differentiation—long-tail terms—are captured from the candidate term set and added to the dedicated keyword set for each semantic unit to fill potential semantic blind spots. This dedicated keyword set for each semantic unit not only improves the depth of semantic representation of the documents but also better aligns with power industry terminology standards, achieving a balance between semantic accuracy and professional applicability.

[0017] Based on the above findings, this embodiment specifically provides a keyword generation scheme, which includes: performing word segmentation on the semantic units corresponding to the document set to obtain a term set, wherein the semantic units are obtained by splitting the power regulations documents in the document set; filtering terms based on the importance of each term in the term set to the document set to form a candidate term set; performing semantic merging and topic aggregation processing on the candidate terms in the candidate term set to obtain a global keyword term set; filtering suitable global keyword terms from the global keyword term set for each semantic unit based on the semantic similarity between the semantic units and the global keyword terms to generate a dedicated keyword term set for each semantic unit; and filtering long-tail terms from the candidate terms based on the degree to which the candidate terms semantically surround the high-frequency terms in the global keyword term set and adding them to the dedicated keyword term set for each semantic unit.

[0018] Based on the above keyword generation scheme, this embodiment specifically provides a keyword generation method and apparatus, which will be described in detail below.

[0019] This application provides a keyword generation method, such as... Figure 1 As shown, the keyword generation method provided in this embodiment may include at least the following steps 101 to 105.

[0020] 101. Perform word segmentation on the semantic units corresponding to the document set to obtain the term set. The semantic units are obtained by splitting the power regulations documents in the document set.

[0021] Electricity regulations documents are standardized technical documents that regulate the operational procedures, technical requirements, safety guidelines, and management standards for power production, construction, operation, maintenance, and testing, ensuring the safe and stable operation of the power system, personnel safety, and reliable equipment operation. In some embodiments, at least one electricity regulation document can be aggregated into a corresponding document set. A corresponding keyword set can then be generated for each electricity regulation document in the document set. This keyword set provides support for quickly locating regulation clauses within the document set, associating knowledge points across documents, and semantic matching by an intelligent document system.

[0022] In some embodiments, after determining the document set, it is necessary to divide the power regulations documents in the document set into semantic units to facilitate knowledge extraction and high-precision retrieval. The process of dividing the power regulations documents in the document set into semantic units may include performing the following steps on each power regulations document in the document set: dividing the current power regulations document into several semantic slices, where a semantic slice is the smallest text unit that independently expresses a complete meaning; and determining the semantic slices obtained from the division as the semantic units corresponding to the current power regulations document.

[0023] Specifically, the methods for dividing the current power regulations document into several semantic slices can include the following two: One method involves preprocessing the current power regulations document to clean up redundant formats and standardize professional terminology; dividing it into macro-units based on structural identifiers such as chapter titles and clause numbers; and then determining each macro-unit as a corresponding semantic slice. The other method involves preprocessing the current power regulations document to clean up redundant formats and standardize professional terminology; dividing it into macro-units based on structural identifiers such as chapter titles and clause numbers; and then, through power industry keyword clustering and semantic association analysis, subdividing the macro-units into independent segments focusing on a single core information (such as operating procedures or safety requirements); after removing semantically broken or overlapping parts, each independent segment is then used as a semantically clear slice. The above two methods can be flexibly selected based on business needs, and this embodiment does not limit the choice.

[0024] In some embodiments, after obtaining the semantic units corresponding to each power regulation document, these semantic units are aggregated to form the semantic units corresponding to the document set. After obtaining the semantic units corresponding to the document set, a step of performing word segmentation on the semantic units corresponding to the document set to obtain a term set is performed, so as to use the term set as the basic data for keyword generation. Specifically, the specific implementation process of performing word segmentation on the semantic units corresponding to the document set to obtain the term set may include the following steps: performing word segmentation on each semantic unit separately; removing stop words from the term items obtained by word segmentation to obtain a valid term set; removing low-frequency words from the valid term set to obtain the term set corresponding to the document set.

[0025] Specifically, the process of segmenting each semantic unit separately can include the following steps for each semantic unit: preprocessing the current semantic unit by cleaning up redundant spaces, special symbols, and meaningless function words, and labeling fixed collocations such as proper nouns, idioms, and terms that cannot be separated to avoid semantic fragmentation; identifying lexical boundaries in the semantic unit based on dictionary matching (forward / reverse maximum matching, bidirectional matching, etc.) or statistical models (such as Hidden Markov Models, CRF, etc.) or deep learning models (such as BERT, etc.) to segment out candidate word units; correcting ambiguous segmentation, over-segmentation / merging, etc. of candidate word units through grammatical rule verification and contextual semantic relevance judgment; and summarizing the fixed collocation terms and the corrected candidate word units into the segmentation result corresponding to the current semantic unit.

[0026] Specifically, to reduce the interference of redundant information without substantial semantic meaning, the word segmentation results are matched with the stop word set, and the stop words in the word segmentation results are removed based on the matching results to obtain the effective word set.

[0027] Specifically, low-frequency terms contribute limited to the core semantics of power regulations documents. Therefore, to reduce interference, low-frequency terms need to be removed from the effective term set. The process of removing low-frequency terms involves performing the following steps for each effective term in the effective term set: determining the number of times the current effective term appears in the document set; determining the total number of times all effective terms in the effective term set appear in the document set; and determining the ratio of the number of occurrences to the total number of occurrences as the relative frequency of the current effective term. If the relative frequency is less than a frequency threshold, it indicates that the current effective term is low-frequency, and therefore, it is removed from the effective term set. After completing the above steps for each effective term, the remaining terms in the effective term set are summarized into a term set. Specifically, the term set can be represented by the following formula: in, Represents a set of terms. This refers to the terms in the term set. Indicates the relative frequency of occurrence of term R. Indicates the frequency threshold. This represents a set of stop words.

[0028] 102. Select terms based on the importance of each term in the term set to the document set, and form a candidate term set.

[0029] In order to enable semantic unit-specific keywords to focus on the core content and key information of power regulations documents and avoid generating irrelevant or secondary keywords, it is necessary to perform a step of filtering keywords based on the importance of each keyword in the keyword set to the document set, and forming a candidate keyword set. The implementation method of this step may include at least the following steps 102A to 102B.

[0030] 102A. For each term, perform the following: Calculate the term frequency-inverse document frequency value as the importance of the current term to the document set based on the frequency of the current term in the document set, the total number of power regulations documents in the document set, and the number of power regulations documents in the document set that contain the current term.

[0031] Specifically, the frequency of occurrence of the current term in the document set can be determined by the following process: determining the number of times the current term appears in the document set, determining the total number of times all terms in the term set appear in the document set, and determining the ratio of the number of occurrences to the total number of occurrences as the frequency of occurrence of the current term.

[0032] Specifically, the calculation process of the term frequency-inverse document frequency value for the current term can be represented by the following formula: Where tfidf(R) represents the term frequency-inverse document frequency value of the current term R, tf(R) represents the frequency of occurrence of the current term R in the document set, df(R) represents the number of power regulations documents in the document set that contain the current term R, and N represents the total number of power regulations documents in the document set.

[0033] 102B. Select terms whose importance is not less than the importance threshold to form a candidate term set.

[0034] Specifically, the lower the importance of a term, the further its semantics deviate from the core semantics of the power regulations document, and the less important it is in the document set. Conversely, the higher the importance of a term, the closer its semantics are to the core semantics of the power regulations document, and the more important it is in the document set. Therefore, in order to use terms close to the core semantics as candidate terms for constructing the keyword term set, terms R with an importance level not less than a certain threshold are selected as candidate terms w, forming the candidate term set.

[0035] Specifically, the candidate term set can be represented by the following formula: V={w∣tfidf(w)>τ} Where V represents the candidate term set, w represents the candidate term w in the candidate term set, tfidf(w) represents the term frequency-inverse document frequency value of the candidate term w, and τ represents the degree threshold.

[0036] 103. Perform semantic merging and topic aggregation processing on the candidate terms in the candidate term set in sequence to obtain the global keyword term set.

[0037] The redundancy and fragmentation of candidate terms in the candidate term set make it difficult to ensure the conciseness and consistency of core concepts in power regulations documents. Therefore, it is necessary to perform semantic merging and topic aggregation processing on the candidate terms in the candidate term set in sequence to obtain a global keyword term set that ensures the conciseness and consistency of core concepts.

[0038] The process of performing semantic merging and topic aggregation on the candidate terms in the candidate term set to obtain the global keyword term set may include the following steps 103A to 103C.

[0039] 103A. Based on the semantic correlation between candidate terms, perform semantic merging on the candidate terms in the candidate term set, and add the merging result to the candidate term set to obtain the target candidate term set.

[0040] The purpose of step 103A is to enrich the terms used for keyword determination. The specific implementation process of step 103A may include the following steps 103A1 to 103A3.

[0041] 103A1. Pair each candidate word with every other candidate word in the candidate word set except itself to form corresponding word pairs.

[0042] The method of generating word pairs one by one in step 103A1 can comprehensively and without omission capture the potential semantic relationships in the candidate word set, avoiding the loss of relationships caused by subjective selection or sampling pairing.

[0043] 103A2. Perform the following for each word pair: Based on the co-occurrence probability of the two candidate words in the current word pair and the independent occurrence probability of each candidate word, determine the semantic correlation between the two candidate words in the current word pair.

[0044] Specifically, the process of determining the co-occurrence probability of two candidate terms in the current word pair may include: determining the total number of times the two candidate terms in the current word pair co-occur in the document set; determining the total number of times any candidate term in the current word pair independently occurs in the document set; and determining the ratio of the total number of co-occurrences to the total number of independent occurrences as the co-occurrence probability of the two candidate terms in the current word pair. Specifically, this process can be expressed by the following formula: Wherein, P(w) i w j ) indicates the candidate term w in the current word pair. i and candidate terms wj The co-occurrence probability, count(w) i w j ) indicates the candidate term w in the current word pair. i and candidate terms w j The total number of times documents co-occur in the document collection, count(w i ) indicates the candidate term w in the current word pair. i The total number of times a word appears independently in a document collection. A higher co-occurrence probability indicates that the candidate word w in the current word pair is more likely to appear. i Appearance, candidate term w j The higher the probability of them appearing together, the stronger their semantic connection.

[0045] Specifically, the process of determining the independent occurrence probability of each candidate word in the current word pair may include performing the following for each candidate word in the current word pair: determining the first total number of times the current candidate word appears independently in the document set, determining the second total number of times all candidate words in the candidate word set appear in the document set, and determining the ratio of the first total number of times to the second total number of times as the independent occurrence probability of the current candidate word.

[0046] Specifically, the process of determining the semantic correlation between two candidate terms in the current word pair based on the co-occurrence probability and the independent occurrence probability of each candidate term in step 103A2 may include: determining the semantic correlation between two candidate terms in the current word pair based on a first formula, wherein the first formula includes: Among them, w i Indicates the candidate terms i and w in the current word pair j NPMI(w) represents the candidate term j in the current word pair. i w j P(w) represents the semantic correlation between candidate term i and candidate term j. i w j P(w) represents the co-occurrence probability of candidate term i and candidate term j. i P(w) represents the independent occurrence probability of candidate term i. j The ) represents the independent occurrence probability of candidate term j. The greater the semantic relevance, the stronger the semantic relevance between the two candidate terms in the current word pair; conversely, the smaller the semantic relevance, the weaker the semantic relevance between the two candidate terms in the current word pair.

[0047] 103A3. Merge two candidate terms in word pairs whose semantic relevance is not less than the relevance threshold.

[0048] For word pairs with a semantic relevance of not less than a relevance threshold, the two candidate terms in the word pair have a strong semantic relevance. Therefore, to avoid redundancy, the two candidate terms in the word pair are merged. This merging process can include the following two methods: First, merge the two candidate terms in the word pair into a single word or compound phrase, where the word or compound phrase is used to represent the semantics of the two candidate terms; Second, determine whether the semantic relevance between the two candidate terms in the word pair is greater than a target threshold. If the target threshold is greater than the relevance threshold, then either of the two candidate terms is used as the merged term; if not, the first merging method is used. The above two methods can be flexibly selected based on business needs, and this embodiment does not limit them.

[0049] After executing steps 103A1 to 103A3, the target candidate term set is obtained. This set includes all candidate terms within the candidate term set, as well as the result of merging two candidate terms from word pairs with a semantic relevance of at least the relevance threshold. The target candidate term set can be represented as: V'=V∪{ (w i w j )∣NPMI(w i w j )>δ} , where V represents the set of candidate terms, V' represents the set of target candidate terms, { (w i w j )∣NPMI(w i w j )>δ} represents the processing result of merging two candidate terms in a word pair whose semantic relevance is not less than the relevance threshold.

[0050] 103B. Cluster the target candidate terms in the target candidate term set into at least one semantic topic cluster.

[0051] To accurately extract representative terms from the target candidate term set, the target candidate terms in the target candidate term set are clustered into at least one semantic topic cluster. The specific implementation process of step 103B may include the following steps: converting each target candidate term into a low-dimensional semantic vector and quantifying the semantic association strength between target candidate terms; then using clustering algorithms (such as K-Means, hierarchical clustering, DBSCAN, etc.) to aggregate semantically similar target candidate terms into clusters based on the similarity of their semantic vectors (such as cosine similarity). During this process, the optimal number of clusters can be determined using contour coefficients, elbow rules, etc. (ensuring at least one semantic topic cluster); finally, semantic induction is performed on each cluster (such as extracting core words within the cluster and calculating the weight of topic keywords), filtering out semantically ambiguous and excessively small invalid clusters, ultimately obtaining at least one semantic topic cluster with clear boundaries and consistent internal semantics. The set of semantic topic clusters clustered from the target candidate term set can be represented as: G={ G1, G2, ..., Gm}, where G represents the set of semantic topic clusters, and Gm represents the m-th semantic topic cluster.

[0052] Specifically, each target candidate word is transformed into a low-dimensional semantic vector using a semantic encoding model (such as Word2Vec, BERT, etc.). This process can be represented by the following formula: e s =fenc (x s e s Let x represent the semantic vector corresponding to the target candidate term s, and fenc represent the semantic encoding function corresponding to the semantic encoding model. s Indicates the target candidate term s.

[0053] 103C. Summarize semantic topic clusters that meet the semantic integration requirements to form a global keyword set.

[0054] The specific implementation process of step 103C may include performing the following steps 103C1 to 103C2 for each semantic topic cluster.

[0055] 103C1. Determine the distribution entropy of the current semantic topic cluster based on the probability of occurrence of each target candidate term in the current semantic topic cluster.

[0056] The distribution entropy of a current semantic topic cluster is an indicator calculated based on the probability of occurrence of each target candidate term within the cluster. It quantifies the uncertainty and dispersion of the distribution of target candidate terms within the current semantic topic cluster. Specifically, if the probability of occurrence of a few target candidate terms within the cluster is extremely high (i.e., the topic focuses on a specific semantic core), the distribution entropy value is low, indicating that the semantic direction of the topic is clear and highly concentrated, and the terms are tightly aggregated around the core semantics. Conversely, if the probability of occurrence of multiple target candidate terms within the cluster is relatively balanced (without a clear core term dominating), the distribution entropy value is high, meaning that the semantic boundaries of the topic are blurred and highly dispersed, and the semantic range covered by the terms is broader without a clear focus. In short, distribution entropy is essentially a quantitative representation of the semantic consistency and focus within a semantic topic cluster; the lower the entropy value, the more condensed the topic, and the higher the entropy value, the more dispersed the topic.

[0057] Specifically, the implementation process of step 103C1 may include: determining the distribution entropy of the current semantic topic cluster based on the second formula, which includes: Among them, G k H(G) represents the current semantic topic cluster. k ) represents the current semantic topic cluster G k The distribution entropy, x f Represents the current semantic topic cluster G k The target candidate terms in p(x) f ) represents the target candidate term x f In the current semantic topic cluster G k The probability of occurrence in.

[0058] Specifically, the process of determining the occurrence probability of a target candidate term in the current semantic topic cluster may include: determining the third total number of times the target candidate term appears independently in the document set, determining the fourth total number of times all target candidate terms in the current semantic topic cluster appear in the document set, and determining the ratio of the third total number to the fourth total number as the independent occurrence probability of the target candidate term.

[0059] 103C2. Select semantic topic clusters whose distribution entropy is not greater than the entropy threshold as semantic topic clusters that meet the requirements of semantic integration, and summarize them to form a global keyword set.

[0060] If the distribution entropy is not greater than the entropy threshold, it indicates a high degree of semantic consistency and focus within the semantic topic cluster, and that the topic is concise. Therefore, semantic topic clusters with distribution entropy not greater than the entropy threshold are selected as semantic topic clusters that meet the requirements for semantic integration, and are aggregated to form a global keyword set. The global keyword set can be represented as: Gvalid={ Gk∣H(Gk)<ϵ}, where Gvalid represents the global keyword set, Gk represents the semantic topic cluster with distribution entropy not greater than the entropy threshold, H(Gk) represents the distribution entropy of the semantic topic cluster Gk, and ϵ represents the entropy threshold.

[0061] 104. Based on the semantic similarity between semantic units and global keyword items, select suitable global keyword items for each semantic unit from the global keyword item set to generate a unique keyword item set for each semantic unit.

[0062] The purpose of step 104 is to accurately select terms from the global keyword list that are highly consistent with the document content for the semantic unit, and to construct its own set of keyword terms. This set of keyword terms not only retains the key features and core expressions unique to the document content of the semantic unit, but also ensures the strong correlation and relevance between the keyword terms and the document content, making the semantic representation of the document more focused and more recognizable. This provides more accurate and efficient basic data support for subsequent tasks such as document retrieval, classification, clustering, and semantic matching, and enables the core information of the document to be captured and utilized more accurately.

[0063] In some embodiments, the specific implementation process of step 104 may include at least performing steps 104A to 104B for each semantic unit distribution.

[0064] 104A. Calculate the semantic score of the current semantic unit and each global keyword item in the global keyword item set respectively.

[0065] The semantic score is used to reflect the semantic similarity between the current semantic unit and the global keyword item. A higher semantic score indicates that the corresponding global keyword item better summarizes the core semantics of the semantic unit. Specifically, the implementation process of step 104 may include: calculating the semantic score between the current semantic unit and the current global keyword item based on the third formula, which includes: Among them, T n Indicates the current semantic unit, This represents the current global keyword item, score(a, T). n ) represents the current semantic unit T n The semantic score of the current global keyword a, tfidf(a) represents the term frequency-inverse document frequency value of the current global keyword a, E(a) represents the semantic vector of the current global keyword a, and E(T) represents the semantic vector of the current global keyword a.n ) represents the current semantic unit T n The semantic vector is denoted by β, which represents the weighting coefficient used to balance word frequency significance and semantic similarity, and β∈[0,1]. Cos represents cosine similarity, and E(⋅) is the semantic embedding encoding function.

[0066] Specifically, the term frequency-inverse document frequency (tfidf(a)) of the current global keyword term 'a' can be calculated using the following formula: Where tfidf(a) represents the term frequency-inverse document frequency value of the current global keyword a, tf(a) represents the frequency of occurrence of the current global keyword a in the document set, df(R) represents the number of power regulations documents in the document set that contain the current global keyword a, and N represents the total number of power regulations documents in the document set.

[0067] Specifically, It can be calculated using the following formula: 104B. Filter out global keyword items with semantic scores not less than the score threshold to form a set of exclusive keyword items corresponding to the current semantic unit.

[0068] Global keywords with semantic scores not less than the score threshold indicate that they can correctly express the core theme of the current semantic unit. Therefore, global keywords with semantic scores not less than the score threshold are selected to form the exclusive keyword set corresponding to the current semantic unit.

[0069] 105. Based on the degree to which candidate terms semantically revolve around high-frequency terms in the global keyword set, long-tail terms are selected from the candidate terms and added to the exclusive keyword set corresponding to each semantic unit.

[0070] If low-frequency but critical terms related to power business (i.e. long-tail terms) are ignored in power regulations documents, it will be difficult for keywords to cover professional power business knowledge. Therefore, it is necessary to select long-tail terms from the candidate term set and add them to the dedicated keyword set corresponding to each semantic unit to make up for possible semantic blind spots.

[0071] In some embodiments, the specific implementation process of step 105 may include the following steps 105A to 105C.

[0072] 105A. Summarize the high-frequency terms in the global keyword set to form a high-frequency term set.

[0073] The process of determining high-frequency terms may include: using term frequency statistics tools (such as Python's collections library, Excel's counting function, etc.) to count the number of times each global keyword appears in the document collection, forming a "term-frequency" correspondence; finally, sorting the terms from high to low frequency, and determining the terms ranked before the target ranking position as high-frequency terms, or finally determining the terms with a frequency greater than a specified threshold as high-frequency terms.

[0074] 105B. For each candidate term, perform the following steps 105B1 to 105B2 respectively.

[0075] 105B1. Based on the average semantic similarity between the current candidate term and the set of high-frequency terms, determine the long-tail compensation factor corresponding to the current candidate term.

[0076] If the current candidate word has a high average similarity to the set of high-frequency terms in the semantic space, it indicates that although the term has a low frequency, it has strong semantic commonality. Therefore, a higher long-tail compensation factor is assigned to increase its weight; conversely, a lower long-tail compensation factor is assigned to decrease its weight. Based on this long-tail compensation factor, the problem of ignoring important low-frequency terms can be solved.

[0077] Specifically, the execution process of step 105B1 may include the following steps: determining the long-tail compensation factor corresponding to the current candidate term based on the fourth formula, which includes: Where η(w) represents the long-tail compensation factor corresponding to the current candidate term w. denoted as the average semantic similarity between the current candidate term w and the set of high-frequency terms, μ represents the average similarity threshold, and λ represents the adjustment coefficient; Specifically, the process of determining the average semantic similarity between the current candidate term w and the set of high-frequency terms may include: in, Let H represent the average semantic similarity between the current candidate term w and the set of high-frequency terms, H represent the total number of high-frequency terms in the set of high-frequency terms, E(w) be the vector of the current candidate term w, and E(hj) be the vector of the j-th high-frequency term in the set of high-frequency terms.

[0078] 105B2. Based on the long-tail compensation factor and the term frequency-inverse document frequency value corresponding to the current candidate term, determine the long-tail score corresponding to the current candidate term.

[0079] Specifically, by combining the term frequency-inverse document frequency (TF-IDF) value (which measures the representativeness of terms) with a long-tail compensation factor (which specifically amplifies the weight of low-frequency but representative terms), the calculated long-tail score can accurately highlight terms that, although they appear infrequently, are highly specific and important to the relevant document or domain. This ensures that the final selected long-tail keywords are not only low-frequency but also crucial, thereby greatly improving the quality of long-tail keywords.

[0080] The specific implementation process of step 105B2 may include the following steps: For each global keyword item in the global keyword item set, perform the following: Based on the fifth formula, determine the long-tail score corresponding to the current candidate keyword item, wherein the fifth formula includes: Among them, score LT η(w) represents the long-tail score corresponding to the current candidate term w, η(w) represents the long-tail compensation factor corresponding to the current candidate term w, and tfidf(w) represents the term frequency-inverse document frequency value corresponding to the current candidate term w.

[0081] Specifically, the term frequency-inverse document frequency value corresponding to the current candidate term can be calculated using the following formula: Where tfidf(w) represents the term frequency-inverse document frequency value of the current candidate term w, tf(w) represents the frequency of occurrence of the current candidate term w in the document set, df(w) represents the number of power regulations documents in the document set that contain the current candidate term w, and N represents the total number of power regulations documents in the document set.

[0082] 105C. Select candidate terms with long-tail scores greater than the long-tail threshold as long-tail terms.

[0083] Candidate terms with a long-tail score greater than the long-tail threshold indicate that although they do not appear frequently, they are key business terms. Therefore, candidate terms with a long-tail score greater than the long-tail threshold are selected as long-tail terms.

[0084] In some embodiments, after filtering out long-tail keywords, the long-tail keywords are added to the dedicated keyword set corresponding to each semantic unit. In this way, the keyword set corresponding to each semantic unit not only strictly conforms to the terminology norms in the power industry, but also achieves high-precision semantic matching with the content of power regulations and documents, thereby providing effective support for subsequent retrieval, knowledge management and content recommendation.

[0085] The keyword generation method provided in this application, upon determining that keywords need to be generated, first obtains semantic units corresponding to a document set. These semantic units are obtained by splitting the power regulations documents within the document set. Then, the semantic units are segmented to obtain a term set. Next, terms are selected based on the importance of each term in the term set to the document set, forming a candidate term set. Then, the candidate terms in the candidate term set are sequentially subjected to semantic merging and topic aggregation processing to obtain a global keyword term set. Then, based on the semantic similarity between the semantic units and the global keyword terms, suitable global keyword terms are selected from the global keyword term set for each semantic unit, generating a dedicated keyword term set for each semantic unit. Finally, based on the degree to which the candidate terms semantically revolve around the high-frequency terms in the global keyword term set, long-tail terms are selected from the candidate terms and added to the dedicated keyword term set corresponding to each semantic unit. As can be seen, the solution provided in this embodiment first focuses keywords on the core content and key information of power regulations documents through term filtering, avoiding the generation of irrelevant or secondary keywords. Then, it integrates scattered terms with overlapping meanings through semantic merging, eliminating redundancy and internal competition between keywords, making the expression more concise and highlighting the core concepts. Subsequently, it mines and establishes semantic relationships between terms through semantic aggregation, constructing semantic thematic term clusters, thereby significantly expanding the breadth and depth of keyword coverage of the document's main theme. The targeted supplementation of long-tail keywords effectively captures low-frequency but business-critical terms in the regulations documents that have professional differentiation but are easily overlooked, significantly improving the uniqueness of the keyword set and the retrieval recall rate. Finally, the keyword set generated by the keyword generation solution provided in this embodiment not only strictly conforms to the terminology norms of the power industry, but also achieves high-precision semantic matching with the content of power regulations documents, thus providing effective support for subsequent retrieval, knowledge management, and content recommendation.

[0086] Furthermore, one embodiment of this application also provides a keyword generation device, such as... Figure 2 As shown, the keyword generation device provided in this embodiment may include at least: The word segmentation module 21 is used to perform word segmentation processing on the semantic units corresponding to the document set to obtain a term set. The semantic units are obtained by splitting the power regulations documents in the document set. The filtering module 22 is used to filter terms based on the importance of each term in the term set to the document set, thereby forming a candidate term set; Processing module 23 is used to sequentially perform semantic merging and topic aggregation processing on the candidate terms in the candidate term set to obtain a global keyword term set; The generation module 24 is used to select suitable global keyword items from the global keyword item set for each semantic unit based on the semantic similarity between the semantic unit and the global keyword items, and generate a unique keyword item set corresponding to each semantic unit. Add module 25, which is used to filter long-tail terms from candidate terms based on the degree to which candidate terms semantically surround high-frequency terms in the global keyword term set, and add them to the dedicated keyword term set corresponding to each semantic unit.

[0087] This application's embodiment first uses term filtering to focus keywords on the core content and key information of power regulations documents, avoiding the generation of irrelevant or secondary keywords. Then, it integrates scattered terms with overlapping meanings through semantic merging, eliminating redundancy and internal competition between keywords, making the expression more concise and highlighting core concepts. Subsequently, semantic aggregation is used to mine and establish semantic relationships between terms, constructing semantic thematic word clusters, thereby significantly expanding the breadth and depth of keyword coverage of the document's main theme. The targeted supplementation of long-tail keywords effectively captures low-frequency but business-critical terms in the regulations documents that are professionally distinctive but easily overlooked, significantly improving the uniqueness of the keyword set and the retrieval recall rate. Finally, the keyword set generated by the keyword generation scheme provided in this embodiment not only strictly conforms to the terminology norms of the power industry but also achieves high-precision semantic matching with the content of power regulations documents, thus providing effective support for subsequent retrieval, knowledge management, and content recommendation.

[0088] In some embodiments of this application, such as Figure 3 As shown, the processing module 23 may include: Processing unit 231 is used to perform semantic merging processing on candidate terms in the candidate term set based on the semantic correlation between candidate terms, and add the merging processing result to the candidate term set to obtain the target candidate term set; Clustering unit 232 is used to cluster the target candidate terms in the target candidate term set into at least one semantic topic cluster; The first filtering unit 233 is used to summarize semantic topic clusters that meet the requirements of semantic integration to form a global keyword item set.

[0089] In some embodiments of this application, such as Figure 3 As shown, the processing unit 231 may include a pairing subunit 2311, a first determining subunit 2312, and a first merging subunit 2313.

[0090] The pairing subunit 2311 is used to pair each candidate word with each candidate word in the candidate word set except itself, to form a corresponding word pair; The first determining subunit 2312 and the first merging subunit 2313 are configured to perform separately for each word pair: The first determining subunit 2312 is used to determine the semantic correlation between two candidate terms in the current word pair based on the co-occurrence probability of the two candidate terms in the current word pair and the independent occurrence probability of each candidate term. The first merging subunit 2313 is used to merge two candidate terms in a word pair with a semantic relevance of not less than the relevance threshold.

[0091] In some embodiments of this application, such as Figure 3 As shown, the first determining subunit 2312 is specifically used to determine the semantic correlation between two candidate terms in the current word pair based on a first formula, the first formula including: Among them, w i Indicates the candidate terms i and w in the current word pair j NPMI(w) represents the candidate term j in the current word pair. i w j P(w) represents the semantic correlation between candidate term i and candidate term j. i w j P(w) represents the co-occurrence probability of candidate term i and candidate term j. i P(w) represents the independent occurrence probability of candidate term i. j ) represents the independent occurrence probability of candidate term j.

[0092] In some embodiments of this application, such as Figure 3 As shown, the first filtering unit 233 may include a second determining subunit 2331 and a second summarizing subunit 2332.

[0093] The second determining subunit 2331 and the second summarizing subunit 2332 are configured to execute separately for each semantic topic cluster: The second determining subunit 2331 is used to determine the distribution entropy of the current semantic topic cluster based on the probability of occurrence of each target candidate word in the current semantic topic cluster. The second aggregation subunit 2332 is used to select semantic topic clusters whose distribution entropy is not greater than the entropy threshold as semantic topic clusters that meet the requirements of semantic integration, and aggregate them to form a global keyword item set.

[0094] In some embodiments of this application, such as Figure 3 As shown, the second determining subunit 2331 is specifically used to determine the distribution entropy of the current semantic topic cluster based on a second formula, which includes: Among them, G k H(G) represents the current semantic topic cluster. k ) represents the current semantic topic cluster G k The distribution entropy, x f Represents the current semantic topic cluster G k The target candidate terms in p(x) f ) represents the target candidate term x f In the current semantic topic cluster G k The probability of occurrence in.

[0095] In some embodiments of this application, such as Figure 3 As shown, the filtering module 22 is specifically used to perform the following for each term: based on the frequency of occurrence of the current term in the document set, the total number of power regulation documents in the document set, and the number of power regulation documents in the document set containing the current term, calculate the term frequency-inverse document frequency value as the importance of the current term to the document set; filter out terms with an importance not less than the degree threshold to form the candidate term set.

[0096] In some embodiments of this application, such as Figure 3 As shown, the generation module 24 may include a calculation unit 241 and a second filtering unit 242. The calculation unit 241 and the second filtering unit 242 are configured to perform distributed operations for each semantic unit: The calculation unit 241 is used to calculate the semantic score of the current semantic unit and each global keyword item in the global keyword item set, and the semantic score is used to reflect the semantic similarity between the current semantic unit and the global keyword item. The second filtering unit 242 is used to filter out global keyword items with semantic scores not less than the score threshold, forming a set of exclusive keyword items corresponding to the current semantic unit.

[0097] In some embodiments of this application, such as Figure 3 As shown, the calculation unit 241 is specifically used to perform the following for each global keyword item in the global keyword item set: calculating the semantic score of the current semantic unit and the current global keyword item based on a third formula, wherein the third formula includes: Among them, T n Indicates the current semantic unit, This represents the current global keyword item, score(a, T). n ) represents the current semantic unit T nThe semantic score of the current global keyword a, tfidf(a) represents the term frequency-inverse document frequency value of the current global keyword a, E(a) represents the semantic vector of the current global keyword a, and E(T) represents the semantic vector of the current global keyword a. n ) represents the current semantic unit T n The semantic vector, β represents the weighting coefficient used to balance word frequency significance and semantic similarity.

[0098] In some embodiments of this application, such as Figure 3 As shown, the adding module 25 may include a summarizing unit 251, a first determining unit 252, a second determining unit 253, and a filtering unit 254.

[0099] The aggregation unit 251 is used to aggregate the high-frequency terms in the global keyword set to form a high-frequency term set; The first determining unit 252, the second determining unit 253, and the filtering unit 254 are configured to perform operations separately for each candidate term: The first determining unit 252 is used to determine the long-tail compensation factor corresponding to the current candidate word based on the average semantic similarity between the current candidate word and the set of high-frequency words; The second determining unit 253 is used to determine the long tail score corresponding to the current candidate term based on the long tail compensation factor and the term frequency-inverse document frequency value corresponding to the current candidate term; The filtering unit 254 is used to filter candidate terms whose long-tail scores are greater than the long-tail threshold as long-tail terms.

[0100] In some embodiments of this application, such as Figure 3 As shown, the first determining unit 252 is specifically used to determine the long-tail compensation factor corresponding to the current candidate term based on the fourth formula, wherein the fourth formula includes: Where η(w) represents the long-tail compensation factor corresponding to the current candidate term w. denoted as the average semantic similarity between the current candidate term w and the set of high-frequency terms, μ represents the average similarity threshold, and λ represents the adjustment coefficient.

[0101] In some embodiments of this application, such as Figure 3 As shown, the second determining unit 253 is specifically used to determine the long-tail score corresponding to the current candidate term based on the fifth formula, wherein the fifth formula includes: Among them, score LTη(w) represents the long-tail score corresponding to the current candidate term w, η(w) represents the long-tail compensation factor corresponding to the current candidate term w, and tfidf(w) represents the term frequency-inverse document frequency value corresponding to the current candidate term w.

[0102] For a detailed explanation of the operation of each functional module in the keyword generation device provided in this application embodiment, please refer to the corresponding detailed explanation of the keyword generation method embodiment above, which will not be repeated here.

[0103] Furthermore, one embodiment of this application also provides a computer-readable storage medium, the storage medium including a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to execute the above-described keyword generation method.

[0104] Furthermore, one embodiment of this application also provides an electronic device, the electronic device comprising: a memory for storing a program; and a processor coupled to the memory for running the program to perform the keyword generation method described above.

[0105] Furthermore, one embodiment of this application also provides a computer program product, the computer program product comprising: a computer program / computer executable instructions, the computer program / computer executable of the above-described keyword generation method.

[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0107] It is understood that the relevant features in the above methods and apparatus can be referenced interchangeably. Furthermore, the terms "first," "second," etc., in the above embodiments are used to distinguish between embodiments and do not represent the superiority or inferiority of any particular embodiment.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0109] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this application.

[0110] In addition, the memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0111] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data cutover device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data cutover device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data cutover device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data cutover device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0117] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0119] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A keyword generation method, characterized in that, The method includes: The semantic units corresponding to the document set are segmented to obtain a term set. The semantic units are obtained by splitting the power regulations documents in the document set. Based on the importance of each term in the term set to the document set, term items are selected to form a candidate term set; Semantic merging and topic aggregation are performed sequentially on the candidate terms in the candidate term set to obtain a global keyword term set; Based on the semantic similarity between semantic units and global keyword items, suitable global keyword items are selected from the global keyword item set for each semantic unit to generate a unique keyword item set for each semantic unit. Based on the degree to which candidate terms semantically surround high-frequency terms in the global keyword set, long-tail terms are selected from the candidate terms and added to the dedicated keyword set corresponding to each semantic unit.

2. The method according to claim 1, characterized in that, The candidate terms in the candidate term set are sequentially subjected to semantic merging and topic aggregation processing to obtain a global keyword term set, including: Based on the semantic correlation between candidate terms, the candidate terms in the candidate term set are semantically merged, and the merged result is added to the candidate term set to obtain the target candidate term set; Cluster the target candidate terms in the target candidate term set into at least one semantic topic cluster; Semantic topic clusters that meet the semantic integration requirements are aggregated to form a global keyword set.

3. The method according to claim 2, characterized in that, Based on the semantic correlation between candidate terms, semantic merging processing is performed on the candidate terms in the candidate term set to obtain the target candidate term set. This includes: pairing each candidate term with each candidate term in the candidate term set except itself to form a corresponding word pair; for each word pair, the following steps are performed: determining the semantic correlation between the two candidate terms in the current word pair based on the co-occurrence probability of the two candidate terms in the current word pair and the independent occurrence probability of each candidate term; and merging processing is performed on the two candidate terms in word pairs with a semantic correlation of not less than the correlation threshold. And / or, Semantic topic clusters that meet the semantic integration requirements are aggregated to form a global keyword set. This includes: for each semantic topic cluster, the following steps are performed: based on the probability of occurrence of each target candidate word in the current semantic topic cluster, the distribution entropy of the current semantic topic cluster is determined; semantic topic clusters whose distribution entropy is not greater than the entropy threshold are selected as semantic topic clusters that meet the semantic integration requirements, and aggregated to form a global keyword set.

4. The method according to claim 3, characterized in that, Based on the co-occurrence probability of two candidate terms in the current word pair and the independent occurrence probability of each candidate term, the semantic correlation between the two candidate terms in the current word pair is determined, including: determining the semantic correlation between the two candidate terms in the current word pair based on a first formula, wherein the first formula includes: Among them, w i Indicates the candidate terms i and w in the current word pair j NPMI(w) represents the candidate term j in the current word pair. i w j P(w) represents the semantic correlation between candidate term i and candidate term j. i w j P(w) represents the co-occurrence probability of candidate term i and candidate term j. i P(w) represents the independent occurrence probability of candidate term i. j ) represents the independent occurrence probability of candidate term j; And / or, Based on the probability of occurrence of each target candidate term in the current semantic topic cluster, the distribution entropy of the current semantic topic cluster is determined, including: determining the distribution entropy of the current semantic topic cluster based on a second formula, wherein the second formula includes: Among them, G k H(G) represents the current semantic topic cluster. k ) represents the current semantic topic cluster G k The distribution entropy, x f Represents the current semantic topic cluster G k The target candidate terms in p(x) f ) represents the target candidate term x f In the current semantic topic cluster G k The probability of occurrence in.

5. The method according to any one of claims 1-4, characterized in that, To form a candidate term set, terms are filtered based on the importance of each term in the term set to the document set. This includes: for each term, the following steps are performed: calculating the term frequency-inverse document frequency value as the importance of the current term to the document set based on the frequency of the current term in the document set, the total number of power regulation documents in the document set, and the number of power regulation documents in the document set containing the current term; and filtering out terms with an importance not less than a threshold value to form the candidate term set. And / or, Based on the semantic similarity between semantic units and global keyword items, suitable global keyword items are selected from the global keyword item set for each semantic unit to generate a unique keyword item set corresponding to each semantic unit. This includes: performing the following operations for each semantic unit: calculating the semantic score between the current semantic unit and each global keyword item in the global keyword item set, where the semantic score is used to reflect the semantic similarity between the current semantic unit and the global keyword items; and selecting global keyword items with semantic scores not less than a score threshold to form a unique keyword item set corresponding to the current semantic unit. And / or, Based on the degree to which candidate terms semantically revolve around high-frequency terms in the global keyword set, long-tail terms are selected from the candidate terms, including: summarizing the high-frequency terms in the global keyword set to form a high-frequency term set; for each candidate term, performing the following: determining the long-tail compensation factor corresponding to the current candidate term based on the average semantic similarity between the current candidate term and the high-frequency term set, and determining the long-tail score corresponding to the current candidate term based on the long-tail compensation factor and the term frequency-inverse document frequency value corresponding to the current candidate term; and selecting candidate terms with long-tail scores greater than the long-tail threshold as long-tail terms.

6. The method according to claim 5, characterized in that, Calculate the semantic score of the current semantic unit and each global keyword item in the global keyword item set, including: for each global keyword item in the global keyword item set, perform the following calculation: calculate the semantic score of the current semantic unit and the current global keyword item based on the third formula, wherein the third formula includes: Among them, T n Indicates the current semantic unit, This represents the current global keyword item, score(a, T). n ) represents the current semantic unit T n The semantic score of the current global keyword a, tfidf(a) represents the term frequency-inverse document frequency value of the current global keyword a, E(a) represents the semantic vector of the current global keyword a, and E(T) represents the semantic vector of the current global keyword a. n ) represents the current semantic unit T n The semantic vector, where β represents the weighting coefficient used to balance word frequency significance and semantic similarity; And / or, Based on the average semantic similarity between the current candidate term and the set of high-frequency terms, the long-tail compensation factor corresponding to the current candidate term is determined, including: determining the long-tail compensation factor corresponding to the current candidate term based on a fourth formula, wherein the fourth formula includes: Where η(w) represents the long-tail compensation factor corresponding to the current candidate term w. λ represents the average semantic similarity between the current candidate term w and the set of high-frequency terms, μ represents the average similarity threshold, and λ represents the adjustment coefficient. And / or, Based on the long-tail compensation factor and the term frequency-inverse document frequency value corresponding to the current candidate term, the long-tail score corresponding to the current candidate term is determined, including: determining the long-tail score corresponding to the current candidate term based on the fifth formula, wherein the fifth formula includes: Among them, score LT η(w) represents the long-tail score corresponding to the current candidate term w, η(w) represents the long-tail compensation factor corresponding to the current candidate term w, and tfidf(w) represents the term frequency-inverse document frequency value corresponding to the current candidate term w.

7. A keyword generation device, characterized in that, The device includes: The word segmentation module is used to perform word segmentation on the semantic units corresponding to the document set to obtain a term set. The semantic units are obtained by splitting the power regulations documents in the document set. The filtering module is used to filter terms based on the importance of each term in the term set to the document set, thereby forming a candidate term set; The processing module is used to sequentially perform semantic merging and topic aggregation processing on the candidate terms in the candidate term set to obtain a global keyword term set; The generation module is used to select suitable global keyword items for each semantic unit from the global keyword item set based on the semantic similarity between the semantic unit and the global keyword items, and generate a unique keyword item set for each semantic unit. An addition module is used to filter long-tail keywords from candidate keywords based on the degree to which candidate keywords semantically surround high-frequency keywords in the global keyword set, and add them to the dedicated keyword set corresponding to each semantic unit.

8. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to execute the keyword generation method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, The electronic device includes: a memory for storing a program; and a processor coupled to the memory for running the program to perform the keyword generation method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes: a computer program / computer-executable instructions, wherein the computer program / computer-executable instructions are the keyword generation method according to any one of claims 1 to 6.

Images