Keyword generation method, device, computer equipment, storage medium and product

By calculating the keyword distinction and inverse frequency in the text set, the methods of screening and clustering generation are solved, and the problems of poor exclusiveness and high labor cost of keyword generation in the existing technology are realized, and the accurate screening of new texts and Efficient keyword generation.

CN114461783BActive Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202210044513.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-14
Publication Date
2024-11-22
Estimated Expiration
2042-01-14

AI Technical Summary

Technical Problem

In the prior art, when generating keywords, the keywords generated by the TF-IDF method are poorly exclusive and require a lot of manual participation, which leads to high costs and difficult to adapt to the screening coverage of new texts.

Method used

By obtaining the first candidate keywords of the text set, their correlation and inverse frequency, calculating the distinction, filtering out the second candidate keywords with strong exclusivity, and clustering them to determine the target keywords to improve the accuracy of screening sex.

Benefits of technology

It improves the exclusivity of keywords, reduces the cost of manual participation, and ensures accurate screening of new texts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114461783B_ABST
    Figure CN114461783B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a keyword generation method, device, computer equipment, storage medium and product, which obtains the first candidate keyword corresponding to the text set, and obtains the relevance and inverse frequency of the first candidate keyword for the text set; calculates the discrimination of the first candidate keyword according to the inverse frequency and relevance; selects the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; clusters the second candidate keywords to obtain the target keyword for text screening. This scheme selects the second candidate keyword with stronger exclusivity from the first candidate keyword according to the relevance and inverse frequency, and then clusters the second candidate keywords to determine the target keyword with stronger exclusivity from the second candidate keywords, so that the target text can be accurately screened when the text is screened based on the target keyword, thereby improving the exclusivity of the generated target keyword and reducing labor costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communication technology, and specifically to a keyword generation method, apparatus, computer equipment, storage medium and product, wherein the storage medium is a computer-readable storage medium and the product is a computer program product. Background Art

[0002] Using keywords to filter text is a common text processing method. For example, if we need to identify a certain type of merchant, we can filter it by matching the merchant name keywords. However, it is time-consuming to directly extract the required keywords from the text, and it is difficult to exhaust all keywords. Moreover, when new text appears, the coverage of filtering text with old keywords will be challenged.

[0003] Keywords can be extracted from texts through the term frequency-inverse document frequency (TF-IDF) method, but the keywords generated by this method have poor exclusivity. Exclusivity can represent the accuracy of screening the target text, so manual participation is often required when generating keywords, that is, first screen keywords through the TF-IDF method, and then manually screen out the required keywords. When new text appears, it is also necessary to use a similar method, first screen keywords through the TF-IDF method, and then manually screen the target keywords from them. The target keywords generated by the above method have poor exclusivity and require a lot of manual participation, which has high labor costs. Summary of the invention

[0004] The embodiments of the present application provide a keyword generation method, apparatus, computer equipment, storage medium and product. The scheme selects second candidate keywords with stronger exclusivity from first candidate keywords according to relevance and inverse class frequency, and then determines target keywords with stronger exclusivity from the second candidate keywords by clustering the second candidate keywords, so that when text screening is performed based on the target keywords, the target text can be accurately screened, thereby improving the exclusivity of the generated target keywords and reducing labor costs.

[0005] A keyword generation method provided in an embodiment of the present application includes:

[0006] Acquire a first candidate keyword corresponding to a text set, and acquire a relevance and an inverse class frequency of the first candidate keyword with respect to the text set;

[0007] Calculating the discrimination degree of the first candidate keyword according to the inverse class frequency and the relevance;

[0008] Selecting second candidate keywords that meet preset conditions from the first candidate keywords according to the discrimination degree;

[0009] The second candidate keywords are clustered to obtain target keywords for text screening.

[0010] Accordingly, the embodiment of the present application also provides a keyword generating device, including:

[0011] An acquisition unit, configured to acquire a first candidate keyword corresponding to a text set, and acquire a relevance and an inverse class frequency of the first candidate keyword with respect to the text set;

[0012] a calculation unit, configured to calculate the discrimination degree of the first candidate keyword according to the inverse class frequency and the relevance;

[0013] A screening unit, configured to screen second candidate keywords that meet a preset condition from the first candidate keywords according to the discrimination degree;

[0014] The clustering unit is used to cluster the second candidate keywords to obtain target keywords for text screening.

[0015] In one embodiment, the acquisition unit includes:

[0016] A statistical subunit, configured to count the text categories to which the texts in the text set containing the first candidate keyword belong, to obtain text category statistics;

[0017] A category acquisition subunit, used to acquire the total number of text categories contained in the text collection;

[0018] The frequency calculation subunit is used to calculate the inverse class frequency of the first candidate keyword for the text set according to the text class statistics and the total number of text classes.

[0019] In one embodiment, the keyword generating device further includes:

[0020] A data acquisition unit, used to acquire an initial text set and initial keywords;

[0021] A text screening unit, configured to perform text screening on the initial text set using the initial keywords to obtain a target text containing the initial keywords;

[0022] The labeling unit is used to label the target text in the initial text sample set with a text category to obtain a text set.

[0023] In one embodiment, the data acquisition unit includes:

[0024] A keyword acquisition subunit, used to acquire a first candidate keyword corresponding to a text sample;

[0025] A keyword screening subunit, configured to screen second candidate keywords from the first candidate keywords according to the relevance and inverse class frequency of the first candidate keywords;

[0026] A sample screening subunit, configured to screen the text sample based on the second candidate keyword to obtain a screening result;

[0027] The index calculation subunit is used to calculate the evaluation index of the second candidate keywords according to the screening result, and select the initial keyword from the second candidate keywords according to the evaluation index.

[0028] In one embodiment, the indicator calculation subunit includes:

[0029] A selection module, configured to select a seed keyword from the second candidate keywords according to the evaluation index;

[0030] An information acquisition module, used to acquire first characteristic information of the first candidate keyword and second characteristic information of the seed keyword;

[0031] A similarity calculation module, configured to calculate the similarity between the first candidate keyword and the seed keyword based on the first feature information and the second feature information;

[0032] A determination module is used to determine the initial keyword from the first candidate keywords according to the seed keyword and the similarity.

[0033] In one embodiment, the acquisition unit includes:

[0034] A frequency information acquisition subunit, used to acquire the word frequency and inverse text frequency of the first candidate keyword;

[0035] The relevance calculation subunit is used to calculate the relevance of the first candidate keyword according to the word frequency and the inverse text frequency.

[0036] In one embodiment, the second candidate keywords include a plurality of keywords, and the clustering unit includes:

[0037] A keyword clustering subunit, configured to cluster the second candidate keywords according to the density reachable relationship between the plurality of second candidate keywords to obtain a target keyword cluster;

[0038] The target keyword determination subunit is used to determine the target keyword from the target keyword cluster.

[0039] Correspondingly, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any keyword generation method provided in the embodiment of the present application.

[0040] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any keyword generation method provided in the embodiment of the present application.

[0041] Accordingly, an embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, any one of the keyword generation methods provided in the embodiments of the present application is implemented.

[0042] The embodiment of the present application obtains the first candidate keyword corresponding to the text set, and obtains the relevance and inverse frequency of the first candidate keyword for the text set; calculates the discrimination of the first candidate keyword according to the inverse frequency and the relevance; selects the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; and clusters the second candidate keywords to obtain the target keyword for text screening.

[0043] This solution selects second candidate keywords with stronger exclusivity from the first candidate keywords based on relevance and inverse class frequency, and then determines target keywords with stronger exclusivity from the second candidate keywords by clustering the second candidate keywords, so that when text screening is performed based on the target keywords, the target text can be accurately screened, thereby improving the exclusivity of the generated target keywords and reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a scene diagram of the keyword generation method provided in the embodiment of the present application;

[0046] Figure 2 is a flowchart of a keyword generation method provided in an embodiment of the present application;

[0047] Figure 3 is another flow chart of the keyword generation method provided in an embodiment of the present application;

[0048] Figure 4is a schematic diagram of a keyword generating device provided in an embodiment of the present application;

[0049] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0051] The embodiment of the present application provides a keyword generation method, device, computer equipment and computer readable storage medium. The keyword generation device can be integrated in a computer equipment, which can be a server or a terminal.

[0052] The terminal may include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.

[0053] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0054] For example, Figure 1 As shown, the computer device obtains the first candidate keyword corresponding to the text set, and obtains the relevance and inverse frequency of the first candidate keyword for the text set; calculates the discrimination of the first candidate keyword according to the inverse frequency and relevance; selects the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; clusters the second candidate keywords to obtain the target keyword for text screening. This solution selects the second candidate keyword with stronger exclusivity from the first candidate keyword according to the relevance and inverse frequency, and then clusters the second candidate keywords to determine the target keyword with stronger exclusivity from the second candidate keywords, so that the target text can be accurately screened when text screening is performed based on the target keyword, thereby improving the exclusivity of the generated target keyword and reducing labor costs.

[0055] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0056] This embodiment will be described from the perspective of a keyword generating device. The keyword generating device may be integrated into a computer device, which may be a server or a terminal.

[0057] The present application embodiment provides a keyword generation method, such as Figure 2 As shown, the specific process of the keyword generation method can be as follows:

[0058] 101. Obtain a first candidate keyword corresponding to a text set, and obtain a relevance and an inverse class frequency of the first candidate keyword with respect to the text set.

[0059] The text set may include multiple texts, and one text may be a sentence, a paragraph or a chapter. The multiple texts include positive texts, and the positive texts represent texts that need to be screened out from the text set.

[0060] The first candidate keywords may include keywords extracted from a text collection.

[0061] The relevance may represent the relevance between a first candidate keyword and the positive text in the text set. The higher the relevance, the more likely it is that the positive text can be screened out according to the first candidate keyword.

[0062] Among them, the inverse class frequency represents the category recognition ability of the first candidate keyword. When the first candidate keyword only appears in the text of one text category, the greater its inverse class frequency, the stronger its category recognition ability, and the more likely it is to filter out the text of this text category based on the first candidate keyword without misselecting the text of other text categories; when a candidate keyword appears in the text of multiple text categories, the smaller its inverse class frequency, the weaker its category recognition ability, and it is difficult to filter out only the text of a certain text category.

[0063] For example, each text included in the text set may be segmented into multiple words, and the words obtained from segmenting each text are used as the first candidate keywords corresponding to the text set.

[0064] The corresponding relevance is calculated for each first candidate keyword, and the relevance can be obtained through a keyword extraction algorithm, which includes a term frequency-inverse document frequency (TF-IDF) algorithm, a text ranking algorithm (TextRank), a latent semantic analysis algorithm (LSA), a latent semantic index (LSI) algorithm, and a latent Dirichlet Allocation (LDA) algorithm.

[0065] The TextRank algorithm is a graph-based ranking algorithm for keyword extraction and document summarization. The TextRank algorithm uses the co-occurrence information (semantics) between words within the text to extract keywords, and can extract the keywords of a given text.

[0066] The TextRank algorithm calculates the relevance of each first candidate keyword based on the following formula, where d is the damping coefficient, which can be reduced to 0.85; WS(V i ) represents the weight of the first candidate keyword i, i.e., relevance; WS(V j ) represents the weight of the first candidate keyword j; ln(V i ) represents a set of first candidate keywords that have a co-occurrence relationship with the first candidate keyword j; w ji Represents the similarity between the first candidate keyword i and the first candidate keyword j.

[0067]

[0068] The LSA algorithm and the LSI algorithm map each text into a vector through the Bag of words (BoW) model, concatenate the word vectors of the first candidate keywords contained in each text in the text set, and obtain a keyword-text matrix. The keyword-text matrix is ​​subjected to a singular value decomposition (SVD) operation. According to the SVD result, the keyword-text matrix is ​​mapped into a low-dimensional space to represent each first candidate keyword as a point in the space composed of k entities. By calculating the similarity between each first candidate keyword and the text, the similarity between each text and each first candidate keyword can be obtained, and the similarity is used as the relevance of the first candidate keyword to the text set.

[0069] The topic corresponding to the text in the text collection is determined by the LDA algorithm. Each topic is represented by n keywords and the corresponding probabilities of the keywords. If the first candidate keyword of the text is consistent with the keyword corresponding to the topic, the probability of the keyword of the topic is determined as the relevance of the first candidate keyword.

[0070] The TF-IDF algorithm calculates the relevance of each first candidate keyword based on its word frequency and inverse text frequency. That is, in one embodiment, the step of “obtaining the relevance of the first candidate keyword to the text set” may specifically include:

[0071] Obtain the word frequency and inverse text frequency of the first candidate keyword;

[0072] According to the word frequency and the inverse text frequency, the relevance of the first candidate keyword is calculated.

[0073] The word frequency may represent the frequency of occurrence of the first candidate keyword in a text of a text category, and may be determined based on the ratio between the total number of words contained in the text of a text category and the number of occurrences of the first candidate keyword in the text of the text category.

[0074] The inverse text frequency may represent the occurrence frequency of the text containing the first candidate keyword in the text set.

[0075] For example, specifically, the total number of first candidate keywords corresponding to the positive text in the text set may be counted, where the first candidate keywords corresponding to the positive text are keywords obtained by performing word segmentation processing on the positive text, and the number of occurrences of each first candidate keyword in the positive text is counted. For each first candidate keyword, the ratio of the number of occurrences in the positive text to the total number of first candidate keywords corresponding to the positive text is calculated to obtain the word frequency of each first candidate keyword, which is TF = number of occurrences of the first candidate keyword in the positive text / total number of words in the positive text.

[0076] The total number of texts contained in the text set is counted, and for each first candidate keyword, the total number of texts containing the first candidate keyword is counted. The inverse text frequency (i.e., IDF) of each first candidate keyword is calculated by the following formula (1). In order to prevent the denominator from being 0, the inverse text frequency can be calculated by formula (2), where a>0, for example, a=1.

[0077]

[0078]

[0079] For each first candidate keyword, the word frequency and inverse text frequency of the candidate keyword are used as the relevance of the first candidate keyword to the text set.

[0080] Optionally, for each candidate keyword, a result obtained by multiplying the word frequency and the inverse text frequency of the first candidate keyword is used as the relevance of the first candidate keyword to the text set.

[0081] When a first candidate keyword has a high correlation and appears in both positive and negative texts, the first candidate keyword is not discriminative, and it is difficult to accurately filter positive samples from the text collection without misselecting negative samples through the first candidate keyword.

[0082] Therefore, for each first candidate keyword, the distinctiveness of the first candidate keyword can be determined by comprehensively considering the relevance and inverse frequency of the first candidate keyword. In one embodiment, the first candidate keyword corresponding to the text set, and the relevance and inverse frequency of each first candidate keyword to the text set can be obtained from a database or blockchain.

[0083] In another embodiment, the inverse class frequency can be calculated based on the total number of text categories to which the text contained in the text sample belongs and the text category statistics corresponding to the text containing the first candidate keyword, that is, the step of "obtaining the inverse class frequency of the first candidate keyword for the text set" can specifically include:

[0084] Counting the text categories to which the texts in the text set containing the first candidate keyword belong, to obtain text category statistics;

[0085] Get the total number of text categories contained in the text collection;

[0086] According to the text category statistics and the total number of text categories, the inverse category frequency of the first candidate keyword for the text collection is calculated.

[0087] The text category statistics may be the number of texts of different text categories in which a first candidate keyword appears, for example, only appears in texts of one text category, or appears in texts of two text categories, and so on.

[0088] The total number of text categories may be the total number of text categories to which the texts contained in the text set belong. For example, if the text set contains two types of text, positive text and negative text, the total number of text categories is 2.

[0089] For example, the total number of text categories of the text set can be determined based on the text categories to which the texts contained in the text set belong; for each first candidate keyword, the text category corresponding to the text in which the first candidate keyword appears in the text set is determined, and the text categories in which the first candidate keyword appears are counted to obtain text category statistics, and the inverse class frequency (i.e., ICF) of each candidate keyword is calculated by the following formula (3). In order to prevent the denominator from being 0, the inverse text frequency can be calculated by formula (4), where a>0, for example, a=1.

[0090]

[0091]

[0092] It is understandable that the text categories of the texts in the text set can be manually annotated, and the texts to be screened out can be manually pre-annotated, or the texts to be screened out can be automatically annotated. That is, in one embodiment, before the step of "obtaining the first candidate keyword corresponding to the text set", the keyword generation method provided in the embodiment of the present application can also specifically include:

[0093] Get the initial text set and the initial keywords;

[0094] Screening the initial text set by using the initial keywords to obtain the target text containing the initial keywords;

[0095] The target text in the initial text sample set is labeled with text categories to obtain a text set.

[0096] The initial text set is a set containing unlabeled texts.

[0097] The initial keywords may include initial keywords used for text screening.

[0098] The target text may include text that needs to be filtered out.

[0099] For example, the specific steps may be to obtain an initial text set and an initial keyword; search for texts containing any initial keywords in the texts contained in the initial text set, and filter out the texts containing any initial keywords to obtain target texts containing the initial keywords, and perform text category annotation on the target texts, for example, add a text label 1 to the target text to indicate that the target text is the text that needs to be filtered out, and obtain a text set.

[0100] In one embodiment, the initial keyword may be a pre-set keyword, which may be obtained from a database or a blockchain. In another embodiment, the initial keyword may be obtained by extracting keywords from a text sample, i.e., the step of “obtaining the initial keyword” may specifically include:

[0101] Obtain the first candidate keyword corresponding to the text sample;

[0102] Selecting a second candidate keyword from the first candidate keyword according to the relevance and inverse frequency of the first candidate keyword;

[0103] Performing text screening on the text sample based on the second candidate keyword to obtain a screening result;

[0104] An evaluation index of the second candidate keyword is calculated according to the screening result, and an initial keyword is selected from the second candidate keyword according to the evaluation index.

[0105] Wherein, the text sample may include texts screened out from the text set, optionally, the text set may be a set after updating (hereinafter referred to as the updated text set), the text sample may be multiple texts in the text set before updating, and the text sample may include all texts in the text set before updating. The text sample set may include texts that need to be screened out, i.e., positive class text samples.

[0106] The first candidate keyword is a keyword obtained by segmenting the text sample into multiple words.

[0107] The screening result may include text screened from the text sample.

[0108] Among them, the evaluation index can represent the screening effect of a first candidate keyword, for example, it can include indicators such as precision, coverage, accuracy, recall, and the harmonic mean of accuracy and recall (also called F1 score, F1_Score).

[0109] If the text sample contains a true positive sample T(u), and the screening result of an alternative keyword is R(u), then the precision is Precision = T(u) ∩ R(u) / R(u); Recall = T(u) ∩ R(u) / T(u); Coverage = R(u) / T(u);

[0110] For example, specifically, the first candidate keyword corresponding to the text sample is obtained, as well as the relevance (the calculation of the relevance of the first candidate keyword can refer to the above-mentioned calculation method of the first candidate related word, which is not repeated here) and the inverse class frequency of each first candidate keyword, and the discrimination of each first candidate keyword is calculated according to the relevance and the inverse class frequency (the specific calculation process can refer to the description in step 102, which is not repeated here), and the first candidate keyword whose discrimination meets the conditions (the discrimination is greater than a preset threshold, or the preset number before the discrimination is sorted, etc.) is screened out as the second candidate keyword.

[0111] The text samples are screened by the second candidate keywords, and texts containing any second candidate keywords are screened out from the text samples to obtain screening results. The screening results may include one or more of the positive text samples and other text samples.

[0112] The evaluation index of the second candidate keyword is calculated based on the number of positive text samples in the screening results of each second candidate keyword, and the number of positive text samples contained in the text sample, and the second candidate keyword whose evaluation index meets the screening conditions (for example, greater than a threshold or less than a threshold, depending on the specific evaluation index) is used as the initial keyword.

[0113] For example, in one embodiment, the precision, coverage, and F1_Score of the second candidate keyword may be calculated, and the second candidate keyword that satisfies all three evaluation indicators is used as the initial keyword.

[0114] In order to improve the screening effect, after screening the second candidate keywords according to the evaluation index, similar keywords can be searched for as initial keywords to improve the coverage of the obtained initial keywords to the positive text samples, that is, the step of "selecting the initial keywords from the second candidate keywords according to the evaluation index" can specifically include:

[0115] Selecting a seed keyword from the second candidate keywords based on the evaluation index;

[0116] Obtaining first characteristic information of the first candidate keyword and second characteristic information of the seed keyword;

[0117] Based on the first feature information and the second feature information, calculating the similarity between the first candidate keyword and the seed keyword;

[0118] An initial keyword is determined from the first candidate keywords according to the seed keyword and the similarity.

[0119] The first feature information may include information characterizing features of the first candidate keyword, for example, the first feature information may be in the form of a feature vector, and the second feature information may include information characterizing features of the seed keyword, for example, the second feature information may be in the form of a feature vector.

[0120] For example, the second candidate keywords whose evaluation indicators meet the conditions can be selected from multiple second candidate keywords as seed keywords. Each seed keyword is encoded, for example, one-hot encoding or word vector embedding to obtain the second feature information of the seed keyword, and the first feature information corresponding to each first candidate keyword is obtained in the same way.

[0121] For each seed keyword, the similarity between the seed keyword and each first candidate keyword is calculated based on its corresponding second feature information and the first feature information corresponding to each first candidate keyword, and a preset number of first candidate keywords with similarities are used as synonyms of the seed keyword, or the first candidate keywords with similarities greater than a preset similarity threshold are used as synonyms of the seed keyword.

[0122] Take each seed keyword and its corresponding synonyms as the initial keywords.

[0123] 102. Calculate the discrimination of the first candidate keyword according to the inverse class frequency and the relevance.

[0124] The discriminability can represent the discriminability of a first candidate keyword, and the ability to distinguish texts of different text categories in a text collection.

[0125] For example, for each first candidate keyword, the discrimination degree of the first candidate keyword may be obtained by multiplying the inverse class frequency and the relevance of the first candidate keyword.

[0126] Optionally, the inverse class frequency and the relevance may be added to obtain the discrimination of the first candidate keyword.

[0127] 103. Filter the second candidate keywords that meet the preset conditions from the first candidate keywords according to the discrimination degree.

[0128] For example, the first candidate keywords with a discrimination degree greater than a preset discrimination degree may be specifically selected as the second candidate keywords. Optionally, a preset number of first candidate keywords with the largest discrimination degree may be selected as the second candidate keywords.

[0129] 104. Cluster the second candidate keywords to obtain target keywords for text screening.

[0130] For example, the second candidate keywords can be clustered by k-means algorithm, Gaussian Mixture Model (GMM), hierarchical clustering algorithm, density-based spatial clustering of applications with noise (DBSCAN), etc., and the target keyword can be determined according to the clustering results.

[0131] Among them, the distance between each second candidate keyword and the cluster center is continuously calculated through the k-means algorithm, the second candidate keywords are classified, and the mean of the second candidate keywords in each cluster is taken, and the cluster center is updated according to the mean until the change of the cluster center tends to be stable. The second candidate keywords can be clustered into multiple keyword clusters, each keyword cluster corresponds to a cluster center, and the distance between each second candidate keyword in the cluster and the cluster center is calculated, and the second candidate keyword whose distance meets the preset distance threshold is determined as the target keyword.

[0132] The DBSCAN algorithm can cluster the second candidate keywords according to the density-reachable relationship between multiple second candidate keywords to obtain a target keyword cluster, and use the second candidate keywords in the target keyword cluster as the target keyword, that is, the step of "clustering the second candidate keywords to obtain the target keyword for text screening" can specifically include:

[0133] Clustering the second candidate keywords according to the density reachable relationship between the plurality of second candidate keywords to obtain a target keyword cluster;

[0134] Identify target keywords from the target keyword cluster.

[0135] Among them, the density reachability relationship includes direct density reachability and indirect density reachability.

[0136] For example, all second candidate keywords may be used as an input data set D=(x1, x2, . . . , xm), and a neighborhood parameter (∈, MinPts) and a keyword distance measurement method may be determined.

[0137] Where ∈ is the neighborhood radius of each core object, MinPts is the neighborhood density threshold, and the object number threshold of the object's ∈-neighborhood. If the number of objects of object O (i.e. keyword O) contains at least MinPts objects, then object O is a core object. If object p is in the ∈-neighborhood of core object q, then p is directly density-reachable from q. If there is an object chain {p1, p2…pn} such that p1=p, pn=q, and pi+1 is in the ∈-neighborhood of pi (i=1,2……n), then p is density-reachable from q (core object).

[0138] The keyword distance measurement methods may include Euclidean distance, Manhattan distance, Chebyshev distance, and Hamming distance.

[0139] The specific clustering process is as follows:

[0140] Step 1: Initialize the core object collection Initialize the number of clusters k = 0, initialize the set of unvisited data sets Γ = D, and divide the clusters

[0141] Step 2: For j=1, 2, ...m, find all core objects according to the following steps a) and b):

[0142] a) Find object x by distance metric j ∈-neighborhood sub-dataset N∈(x j );

[0143] b) If the number of samples in the subsample set satisfies |N∈(x j )|≥MinPts, the sample x j Add core object set: Ω=Ω∪{x j}.

[0144] Step 3: If the core object collection The algorithm ends, otherwise it goes to step 4.

[0145] Step 4: Randomly select a core object o from the core object set Ω, initialize the current cluster core object queue Ωcur = {o}, initialize the category sequence k = k + 1, and initialize the current cluster object set C k ={o}, update the unvisited sample set Γ=Γ-{o}.

[0146] Step 5: If the current cluster core object queue The current cluster C k After generation, update the cluster partition C = {C1, C2, ..., C k}, update the core object set Ω = Ω-C k , go to step 3.

[0147] Step 6: Take out a core object o′ from the current cluster core object queue Ωcur, find all ∈-neighborhood sub-datasets N∈(o′) through the neighborhood distance threshold ∈, let Δ=N∈(o′)∩Γ, and update the current cluster object set C k=C k ∪Δ, update the unvisited sample set Γ=Γ-Δ, update the current cluster core object queue Ωcur=Ωcur∪(Δ∩Ω)-o′, and go to step 5.

[0148] The output result of the algorithm is: target keyword cluster division C = {C1, C2, ..., C k}.

[0149] The second candidate keyword in the target keyword cluster is used as the target keyword. Optionally, the second candidate keyword in the target keyword cluster that is different from the initial keyword is used as the target keyword. The target keyword and the initial keyword are used together as keywords for text screening.

[0150] For example, when it is necessary to identify a certain type of merchant or a certain type of text, the above process can generate keywords (i.e., target keywords and initial keywords) for users to filter this type of merchant or this type of text. Afterwards, this type of merchant or this type of text can be filtered from the database based on the generated keywords.

[0151] As can be seen from the above, the embodiment of the present application obtains the first candidate keyword corresponding to the text set, and obtains the relevance and inverse frequency of the first candidate keyword for the text set; calculates the discrimination of the first candidate keyword according to the inverse frequency and the relevance; selects the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; clusters the second candidate keywords to obtain the target keyword for text screening.

[0152] This solution selects second candidate keywords with stronger exclusivity from the first candidate keywords based on relevance and inverse class frequency, and then determines target keywords with stronger exclusivity from the second candidate keywords by clustering the second candidate keywords, so that when text screening is performed based on the target keywords, the target text can be accurately screened, thereby improving the exclusivity of the generated target keywords and reducing labor costs.

[0153] Based on the above embodiments, further detailed description will be given below with examples.

[0154] This embodiment will be described from the perspective of a keyword generating device, which may be specifically integrated in a server.

[0155] The embodiment of the present application provides a keyword generation method, which includes two stages: a keyword generation stage and a keyword update and iteration stage. Figure 3 As shown, the specific process of the keyword generation method can be as follows:

[0156] 1. Keyword generation stage.

[0157] 201. The server obtains a pre-update text set, and performs word segmentation processing on text samples in the pre-update text set to obtain a plurality of first candidate keywords.

[0158] The pre-update sample set may be a text set without any new text added, and the pre-update sample set includes positive text samples and negative text samples, and the positive text samples indicate that the text samples are needed from the pre-update sample set.

[0159] For example, the server may perform word segmentation processing on each text sample included in the pre-update text set, divide each text sample into multiple words, and use the words obtained from the segmentation of each text sample as the first candidate keyword corresponding to the pre-update text set.

[0160] 202. The server calculates the discrimination degree of each first candidate keyword according to the word frequency, inverse text frequency and inverse category frequency of the first candidate keyword.

[0161] For example, the server may specifically count the total number of first candidate keywords corresponding to the positive text samples in the text set before the update, count the number of occurrences of each first candidate keyword in the positive text samples, and for each first candidate keyword, calculate the number of occurrences in the positive text samples and the ratio of the number of occurrences to the total number of first candidate keywords corresponding to the positive text samples to obtain the word frequency of each first candidate keyword, which is TF = number of occurrences of the first candidate keyword in the positive text samples / total number of words in the positive text samples.

[0162] The server counts the total number of text samples contained in the text set before the update, and for each first candidate keyword, counts the total number of texts containing the first candidate keyword in the text set before the update, and calculates the inverse text frequency (i.e., IDF) of each first candidate keyword using the following formula (5).

[0163]

[0164] Since the text set before updating contains positive text samples and negative text samples, the total number of text categories is 2; for each first candidate keyword, determine the text category corresponding to the text sample in which the first candidate keyword appears in the text set, and count the text categories in which the first candidate keyword appears to obtain text category statistics. Because it is possible that only the positive text sample or the negative text sample contains the first candidate keyword, or it is possible that both the positive text sample and the negative text sample contain the first candidate keyword, the text category statistics of each first candidate keyword is 1 or 2. The inverse class frequency (i.e., ICF) of each candidate keyword is calculated by the following formula (6).

[0165]

[0166] For each first candidate keyword, the server multiplies the inverse class frequency of the first candidate keyword by the relevance to obtain the discrimination degree of each first candidate keyword.

[0167] 203. The server selects a second candidate keyword from the first candidate keyword, and performs text screening on the text sample based on the second candidate keyword to obtain a screening result.

[0168] For example, the server may specifically select the first candidate keywords whose discrimination degree meets the condition (the discrimination degree is greater than a preset threshold value, or a preset number before discrimination degree sorting, etc.) as the second candidate keywords.

[0169] The server filters the text samples in the pre-update text set using the second candidate keyword, and filters out text samples containing any second candidate keyword from the pre-update text set to obtain a screening result, which may include one or more of a positive text sample and a negative text sample.

[0170] 204. The server selects a seed keyword from the second candidate keywords according to the evaluation index of the second candidate keywords, and determines an initial keyword from a keyword that is a synonym of the seed keyword in the first candidate keywords.

[0171] For example, the evaluation indicators of the second candidate keyword, namely, precision, coverage, and F1_Score, can be calculated based on the number of positive text samples in the screening results of each second candidate keyword and the number of positive text samples included in the text set before updating, and the second candidate keyword whose evaluation indicators meet the screening conditions (for example, greater than a threshold or less than a threshold, depending on the specific evaluation indicators) is used as the seed keyword.

[0172] Optionally, the seed keywords may be screened from the second candidate keywords through manual participation.

[0173] In order to increase the coverage of the positive text samples by the obtained initial keywords and improve the screening effect, the synonyms of the seed keywords can be searched in the first candidate keywords, and the seed keywords and the synonyms of the seed keywords can be used as the initial keywords. Specifically, the seed keywords are mapped to the feature space through word vector embedding to obtain the corresponding second feature vector and the first feature vector of the first candidate keyword. For each seed keyword, according to the second feature vector of the seed keyword and the first feature vector of the first candidate keyword, and through the distance between the first feature vector and the second feature vector, a preset number of first candidate keywords that are most adjacent to the seed keyword and the seed keyword are used as the initial keywords.

[0174] To ensure the exclusivity of keywords, the precision, F1-Score, and coverage rate of each synonym of the seed keyword can be calculated, and the synonyms with precision, F1-Score, and coverage rate within a preset range are used as the initial keywords.

[0175] II. Keyword update and iteration stage.

[0176] 205. When there is new text in the text library, the server obtains the initial text set and performs text category annotation on the target text in the initial text set based on the initial keywords to obtain the text set.

[0177] For example, specifically, when there is new text in the text library, the server obtains the initial text set containing the new text and obtains the initial keywords through the initial text set; in the texts included in the initial text set, it queries the texts containing any initial keyword and filters out the texts containing any initial keyword to obtain the target text containing the initial keyword, and performs text category annotation on the target text. For example, it adds the text label 1 to the target text, indicating that the target text is the text to be filtered out, to obtain the text set.

[0178] 206. The server obtains the first candidate keywords corresponding to the text set, and calculates the discrimination degree of each first candidate keyword according to the relevance and inverse class frequency of each first candidate keyword for the text set.

[0179] For example, specifically, the server performs word segmentation on each text included in the text set, cuts each text into multiple words, and uses the words obtained by cutting each text as the first candidate keywords corresponding to the text set.

[0180] Referring to the calculation of the discrimination degree of the first alternative keyword in step 202, calculate the discrimination degree of the first candidate keyword.

[0181] 207. The server filters out the second candidate keywords that meet the preset conditions from the first candidate keywords according to the discrimination degree.

[0182] For example, specifically, the server filters out the first candidate keywords with a discrimination degree greater than the preset discrimination degree as the second candidate keywords. Optionally, the preset number of first candidate keywords with the largest discrimination degree can also be used as the second candidate keywords.

[0183] Optionally, the first candidate keywords can also be filtered for stop words, filtering out the stop words of the first candidate keywords. Stop words can include words that have no meaning compared with other words and words that are difficult to filter out the correct text based on that word. For example, "de", "shi", "ya", "the", and "want", etc.

[0184] 208. The server clusters the second candidate keywords to obtain target keywords for text screening.

[0185] For example, the second candidate keywords may be clustered by using a DBSCAN algorithm according to the density between multiple second candidate keywords to obtain a target keyword cluster.

[0186] The second candidate keyword in the target keyword cluster is used as the target keyword. Optionally, the second candidate keyword in the target keyword cluster that is different from the initial keyword is used as the target keyword, and the target keyword and the initial keyword are used together as keywords for text screening.

[0187] Optionally, the accuracy and coverage of the second candidate keyword in the target keyword cluster for the text library are calculated. When the corresponding accuracy and coverage are within a preset range, the second candidate keyword is used as the target keyword. No human participation is required in the keyword update iteration stage, which reduces labor costs and improves the timeliness of keyword update iteration.

[0188] As can be seen from the above, the server of the embodiment of the present application obtains a text set before updating and performs word segmentation processing on the text samples in the text set before updating to obtain multiple first candidate keywords; calculates the discrimination degree of the first candidate keywords according to the word frequency, inverse text frequency and inverse class frequency of each first candidate keyword; selects a second candidate keyword from the first candidate keyword, and performs text screening on the text sample based on the second candidate keyword to obtain a screening result; selects a seed keyword from the second candidate keyword according to the evaluation index of the second candidate keyword, and determines the initial keyword from the keyword that is a synonym of the seed keyword in the first candidate keyword; when there is a new text in the text library, the server obtains the initial text set, and performs text category annotation on the target text in the initial text set based on the initial keyword to obtain a text set; obtains the first candidate keyword corresponding to the text set, and calculates the discrimination degree of the first candidate keyword according to the relevance and inverse class frequency of each first candidate keyword to the text set; screens the second candidate keywords that meet the preset conditions from the first candidate keywords according to the discrimination degree; clusters the second candidate keywords to obtain the target keyword for text screening.

[0189] This solution selects second candidate keywords with stronger exclusivity from the first candidate keywords based on relevance and inverse class frequency, and then determines target keywords with stronger exclusivity from the second candidate keywords by clustering the second candidate keywords, so that when text screening is performed based on the target keywords, the target text can be accurately screened, thereby improving the exclusivity of the generated target keywords and reducing labor costs.

[0190] In order to facilitate better implementation of the keyword generation method provided in the embodiment of the present application, a keyword generation device is also provided in one embodiment. The meaning of the nouns is the same as in the above keyword generation method, and the specific implementation details can refer to the description in the method embodiment.

[0191] The keyword generating device can be integrated into a computer device, such as Figure 4 As shown, the keyword generating device may include: an acquisition unit 301, a calculation unit 302, a screening unit 303 and a clustering unit 304, which are specifically as follows:

[0192] Accordingly, the embodiment of the present application also provides a keyword generating device, including:

[0193] (1) An acquisition unit 301 is used to acquire a first candidate keyword corresponding to a text set, and acquire a relevance and an inverse class frequency of the first candidate keyword with respect to the text set.

[0194] In one embodiment, the acquisition unit 301 includes a statistical subunit, a category acquisition subunit, and a frequency calculation subunit, specifically:

[0195] A statistical subunit, used for counting the text categories to which the texts in the text set containing the first candidate keyword belong, to obtain text category statistics;

[0196] A category acquisition subunit is used to acquire the total number of text categories contained in a text collection;

[0197] The frequency calculation subunit is used to calculate the inverse class frequency of the first candidate keyword for the text set according to the text class statistics and the total number of text classes.

[0198] In one embodiment, the keyword generation device further includes a data acquisition unit, a text screening unit and a marking unit, specifically:

[0199] A data acquisition unit, used to acquire an initial text set and initial keywords;

[0200] A text screening unit, used to screen the initial text set by using the initial keywords to obtain a target text containing the initial keywords;

[0201] The labeling unit is used to label the target text in the initial text sample set with a text category to obtain a text set.

[0202] In one embodiment, the data acquisition unit includes a keyword acquisition subunit, a keyword screening subunit, a sample screening subunit, and an indicator calculation subunit, specifically:

[0203] A keyword acquisition subunit, used to acquire a first candidate keyword corresponding to a text sample;

[0204] A keyword screening subunit, configured to screen a second candidate keyword from the first candidate keyword according to the relevance and inverse frequency of the first candidate keyword;

[0205] A sample screening subunit, used to screen the text sample based on the second candidate keyword to obtain a screening result;

[0206] The index calculation subunit is used to calculate the evaluation index of the second candidate keywords according to the screening result, and select the initial keyword from the second candidate keywords according to the evaluation index.

[0207] In one embodiment, the indicator calculation subunit includes a selection module, an information acquisition module, a similarity calculation module and a determination module, specifically:

[0208] A selection module, used for selecting a seed keyword from the second candidate keywords according to the evaluation index;

[0209] An information acquisition module, used to acquire first characteristic information of the first candidate keyword and second characteristic information of the seed keyword;

[0210] A similarity calculation module, used to calculate the similarity between the first candidate keyword and the seed keyword based on the first feature information and the second feature information;

[0211] The determination module is used to determine the initial keyword from the first candidate keywords according to the seed keyword and the similarity.

[0212] In one embodiment, the acquisition unit 301 includes a frequency information acquisition subunit and a relevance calculation subunit, specifically:

[0213] A frequency information acquisition subunit, used to acquire the word frequency and inverse text frequency of the first candidate keyword;

[0214] The relevance calculation subunit is used to calculate the relevance of the first candidate keyword according to the word frequency and the inverse text frequency.

[0215] (2) A calculation unit 302, configured to calculate the discrimination of the first candidate keyword according to the inverse class frequency and the relevance.

[0216] (3) A screening unit 303, configured to screen second candidate keywords that meet a preset condition from the first candidate keywords according to the degree of discrimination.

[0217] (4) A clustering unit 304, which is used to cluster the second candidate keywords to obtain target keywords for text screening.

[0218] In one embodiment, the second candidate keywords include multiple ones, and the clustering unit 304 includes, specifically:

[0219] A keyword clustering subunit, used for clustering the second candidate keywords according to the density reachable relationship between the plurality of second candidate keywords to obtain a target keyword cluster;

[0220] The target keyword determination subunit is used to determine the target keyword from the target keyword cluster.

[0221] As can be seen from the above, the keyword generation device of the embodiment of the present application obtains the first candidate keyword corresponding to the text set through the acquisition unit 301, and obtains the relevance and inverse frequency of the first candidate keyword for the text set; the calculation unit 302 calculates the discrimination of the first candidate keyword according to the inverse frequency and the relevance; the screening unit 303 screens the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; finally, the clustering unit 304 clusters the second candidate keywords to obtain the target keyword for text screening.

[0222] This solution selects second candidate keywords with stronger exclusivity from the first candidate keywords based on relevance and inverse class frequency, and then determines target keywords with stronger exclusivity from the second candidate keywords by clustering the second candidate keywords, so that when text screening is performed based on the target keywords, the target text can be accurately screened, thereby improving the exclusivity of the generated target keywords and reducing labor costs.

[0223] The present application also provides a computer device, which may be a terminal or a server. Figure 5 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0224] The computer device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0225] The processor 1001 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, computer program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0226] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0227] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0228] The computer device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0229] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:

[0230] By obtaining a first candidate keyword corresponding to the text set, and obtaining a relevance and an inverse class frequency of the first candidate keyword with respect to the text set;

[0231] Calculate the discrimination of the first candidate keyword according to the inverse class frequency and relevance;

[0232] Selecting second candidate keywords that meet preset conditions from the first candidate keywords according to the discrimination degree;

[0233] The second candidate keywords are clustered to obtain target keywords for text screening.

[0234] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0235] As can be seen from the above, the computer device of the embodiment of the present application can obtain the first candidate keyword corresponding to the text set, and obtain the relevance and inverse frequency of the first candidate keyword for the text set; calculate the discrimination of the first candidate keyword according to the inverse frequency and relevance; select the second candidate keyword that meets the preset conditions from the first candidate keyword according to the discrimination; cluster the second candidate keywords to obtain the target keyword for text screening. This solution selects the second candidate keyword with stronger exclusivity from the first candidate keyword according to the relevance and inverse frequency, and then clusters the second candidate keywords to determine the target keyword with stronger exclusivity from the second candidate keywords, so that the target text can be accurately screened when text screening is performed based on the target keyword, thereby improving the exclusivity of the generated target keyword and reducing labor costs.

[0236] According to one aspect of the present application, a computer program product is provided, the computer program product includes a computer program, the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations in the above embodiments.

[0237] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0238] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any keyword generation method provided in the embodiment of the present application.

[0239] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0240] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0241] Since the computer program stored in the computer-readable storage medium can execute any keyword generation method provided in the embodiments of the present application, the beneficial effects that can be achieved by any keyword generation method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0242] The above is a detailed introduction to a keyword generation method, device, computer equipment, computer-readable storage medium and computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A keyword generation method, characterized in that: include: Obtaining an initial text set and obtaining a first candidate keyword corresponding to the text sample; Filtering second candidate keywords from the first candidate keywords according to the relevance and inverse frequency of the first candidate keywords; Performing text screening on the text sample based on the second candidate keyword to obtain a screening result; Calculating an evaluation index of the second candidate keywords according to the screening result, and selecting an initial keyword from the second candidate keywords according to the evaluation index; Performing text screening on the initial text set using the initial keywords to obtain a target text containing the initial keywords; Performing text category labeling on the target text in the initial text set to obtain a text set; Acquire a first candidate keyword corresponding to a text set, and acquire a relevance and an inverse class frequency of the first candidate keyword with respect to the text set; Calculating the discrimination degree of the first candidate keyword according to the inverse class frequency and the relevance; Selecting second candidate keywords that meet preset conditions from the first candidate keywords according to the discrimination degree; The second candidate keywords are clustered to obtain target keywords for text screening.

2. The method according to claim 1, characterized in that The selecting the initial keyword from the second candidate keywords by using the evaluation index includes: Selecting a seed keyword from the second candidate keywords according to the evaluation index; Acquire first characteristic information of the first candidate keyword and second characteristic information of the seed keyword; Based on the first feature information and the second feature information, calculating the similarity between the first candidate keyword and the seed keyword; The initial keyword is determined from the first candidate keywords according to the seed keyword and the similarity.

3. The method according to claim 1, characterized in that The obtaining of the relevance of the first candidate keyword to the text set includes: Obtaining the word frequency and inverse text frequency of the first candidate keyword; The relevance of the first candidate keyword is calculated according to the word frequency and the inverse text frequency.

4. The method according to any one of claims 1 to 3, characterized in that: The second candidate keywords include a plurality of keywords, and clustering the second candidate keywords to obtain target keywords for text screening includes: Clustering the second candidate keywords according to the density reachable relationship between the plurality of second candidate keywords to obtain a target keyword cluster; The target keyword is determined from the target keyword cluster.

5. A keyword generating device, characterized in that: include: A data acquisition unit, used to acquire an initial text set and a first candidate keyword corresponding to the text sample; A keyword screening subunit, configured to screen second candidate keywords from the first candidate keywords according to the relevance and inverse class frequency of the first candidate keywords; A sample screening subunit, configured to screen the text sample based on the second candidate keyword to obtain a screening result; an index calculation subunit, configured to calculate an evaluation index of the second candidate keywords according to the screening result, and select an initial keyword from the second candidate keywords according to the evaluation index; A text screening unit, configured to perform text screening on the initial text set using the initial keywords to obtain a target text containing the initial keywords; A labeling unit, used for labeling the target text in the initial text set with a text category to obtain a text set; An acquisition unit, configured to acquire a first candidate keyword corresponding to a text set, and acquire a relevance and an inverse class frequency of the first candidate keyword with respect to the text set; a calculation unit, configured to calculate the discrimination degree of the first candidate keyword according to the inverse class frequency and the relevance; A screening unit, configured to screen second candidate keywords that meet a preset condition from the first candidate keywords according to the discrimination degree; The clustering unit is used to cluster the second candidate keywords to obtain target keywords for text screening.

6. A computer device, characterized in that: The method comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the keyword generation method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the keyword generation method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the keyword generating method according to any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Internet website classification method and device

    CN106156372A

  • Method and device for extracting keywords of business field

    CN108153781A

  • A method for extracting a target keyword from a text

    CN109918657A

  • Keyword extraction method, keyword extraction device and computer readable storage medium

    CN110008401A

  • Text keyword extraction method and device and computer readable storage medium

    CN111950261A