A keyword processing method and device, electronic equipment and storage medium

Abstract

The application provides a keyword processing method and device, electronic equipment and computer storage medium, the method comprises: obtaining the corpus of public opinion in a set time span; determining a plurality of public opinion keywords according to the corpus; determining a reference time sequence corresponding to the plurality of public opinion keywords, determining the weight time sequence of each public opinion keyword in the plurality of public opinion keywords based on the reference time sequence; the reference time sequence represents the time sequence corresponding to the total editing times of the plurality of public opinion keywords; clustering the plurality of public opinion keywords according to the weight time sequence of each public opinion keyword to obtain the clustering result of the plurality of public opinion keywords; in the related art, only the semantic method is used to identify the public opinion keywords in the corpus and cluster, and the time information of the public opinion keywords is not considered, so it is difficult to determine the evolution mode of the public opinion keywords, and the effectiveness of subsequent analysis and monitoring of public opinion events is reduced.

Description

Technical Field

[0001] This application relates to the field of public opinion analysis technology, and in particular to a keyword processing method, apparatus, electronic device, and computer storage medium. Background Technology

[0002] With the rapid development of the internet, online media, as a new form of information dissemination, is widely used to create and edit articles related to major public opinion events such as social news, natural disasters, and entertainment incidents. Taking Wikipedia articles as an example, the edit history of each article stores a large number of edits; the keywords in the edit history related to public opinion events change over time, which we consider keyword evolution. To facilitate subsequent analysis and monitoring of public opinion events, clustering the keywords in the edit history is crucial. However, current technologies rely solely on semantic methods such as Word2vec and TextRank to identify and cluster public opinion keywords in corpora. This clustering method struggles to determine the evolutionary patterns of these keywords, reducing the effectiveness of subsequent analysis and monitoring of public opinion events. Summary of the Invention

[0003] This application provides a keyword processing method, apparatus, electronic device, and computer storage medium; it can aggregate keywords with similar outbreak patterns in the editing history of public opinion events, and effectively determine the temporal evolution pattern of keywords in public opinion events.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides a keyword processing method, the method comprising:

[0006] Acquire corpus of public opinion within a set time span; determine multiple public opinion keywords based on the corpus;

[0007] A baseline time series corresponding to the plurality of public opinion keywords is determined, and based on the baseline time series, a weighted time series for each of the plurality of public opinion keywords is determined; the baseline time series represents the time series corresponding to the total number of edits of the plurality of public opinion keywords.

[0008] Based on the weight time series of each public opinion keyword, the multiple public opinion keywords are clustered to obtain the clustering results of the multiple public opinion keywords.

[0009] In some embodiments, determining the weight time series of each of the plurality of public opinion keywords based on the baseline time series includes:

[0010] The reference time series is decomposed into components to obtain the individual component components of the reference time series; each component component represents a time series component including a single burst.

[0011] By linearly combining the components of the baseline time series, the weighted time series of each public opinion keyword is obtained.

[0012] In some embodiments, the component decomposition of the reference time series includes:

[0013] The baseline time series is decomposed into components using a time series component decomposition model.

[0014] In some embodiments, the linear combination of the components of the reference time series includes:

[0015] The weight determination model is used to linearly combine the components of the benchmark time series; the weight determination model is used to determine the weights of the components of the benchmark time series.

[0016] In some embodiments, the training method of the weight determination model includes the following steps:

[0017] Input the sample set of public opinion keywords into the weight determination model;

[0018] The weight determination model is used to process the public opinion keyword sample set to obtain the predicted weight of each keyword sample in the public opinion keyword sample set;

[0019] Based on the prediction weight and annotation weight of each keyword sample in the public opinion keyword sample set, the weights are adjusted to determine the network parameter values ​​of the model.

[0020] In some embodiments, clustering the plurality of public opinion keywords to obtain the clustering results of the plurality of public opinion keywords includes:

[0021] Determine the similarity between the weighted time series of each of the multiple public opinion keywords;

[0022] Based on the similarity, the clustering results of the multiple public opinion keywords are obtained.

[0023] In some embodiments, the method further includes:

[0024] After obtaining the clustering results of the multiple public opinion keywords, the evolution of the clustering results within a set time span is visualized.

[0025] This application provides a keyword processing device, which includes a first determining module, a second determining module, and a clustering module, wherein...

[0026] The first determining module is used to acquire corpus of public opinion within a set time span; and to determine multiple public opinion keywords based on the corpus.

[0027] The second determining module is used to determine the baseline time series corresponding to the plurality of public opinion keywords, and based on the baseline time series, determine the weight time series of each of the plurality of public opinion keywords; the baseline time series represents the time series corresponding to the total number of edits of the plurality of public opinion keywords;

[0028] The clustering module is used to cluster the multiple public opinion keywords according to the weight time series of each public opinion keyword, and obtain the clustering result of the multiple public opinion keywords.

[0029] This application provides an electronic device, the device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the keyword processing method provided by one or more of the foregoing technical solutions.

[0030] This application provides a computer storage medium storing a computer program; when the computer program is executed, it can implement the keyword processing method provided by one or more of the aforementioned technical solutions.

[0031] This application provides a keyword processing method, apparatus, electronic device, and computer storage medium. The method includes: acquiring corpus of public opinion within a set time span; determining multiple public opinion keywords based on the corpus; determining a baseline time series corresponding to the multiple public opinion keywords; determining a weighted time series for each of the multiple public opinion keywords based on the baseline time series; the baseline time series representing a time series corresponding to the total number of edits of the multiple public opinion keywords; and clustering the multiple public opinion keywords according to the weighted time series of each public opinion keyword to obtain a clustering result for the multiple public opinion keywords. It can be seen that in this application embodiment, after obtaining multiple public opinion keywords based on the public opinion corpus, the time series corresponding to the total number of edits of the multiple public opinion keywords is used as the baseline time series, and the weighted time series of each public opinion keyword is determined based on the baseline time series. Furthermore, clustering the multiple public opinion keywords using the weighted time series of each public opinion keyword allows for the determination of the evolution pattern of the public opinion keywords from a time perspective, facilitating subsequent analysis and monitoring of public opinion events. Attached Figure Description

[0032] Figure 1a This is a flowchart of a keyword processing method according to an embodiment of this application;

[0033] Figure 1b This is a schematic diagram of a linear model according to an embodiment of this application;

[0034] Figure 1c This is a schematic diagram of a multi-Gaussian model according to an embodiment of this application;

[0035] Figure 1d This is a schematic diagram illustrating the component decomposition of a time series using a linear model, as described in an embodiment of this application.

[0036] Figure 1e This is a schematic diagram (2) illustrating the component decomposition of a time series using a linear model, as described in an embodiment of this application.

[0037] Figure 1f This is a schematic diagram of component decomposition of a time series using a linear model, as described in an embodiment of this application. Figure 3 ;

[0038] Figure 1g This is a schematic diagram illustrating the fitting of a baseline time series using a linear model, as described in an embodiment of this application.

[0039] Figure 1h This is a schematic diagram (2) illustrating the fitting of a baseline time series using a linear model, as described in an embodiment of this application.

[0040] Figure 1i This is a schematic diagram of fitting a baseline time series using a linear model, as described in an embodiment of this application. Figure 3 ;

[0041] Figure 2a This is a schematic diagram illustrating the various components in the baseline time series in the embodiments of this application;

[0042] Figure 2b This is a schematic diagram illustrating the clustering of public opinion keywords into various keyword clusters in an embodiment of this application;

[0043] Figure 2c This is a schematic diagram of the time series of representative keywords in each keyword cluster in the embodiments of this application;

[0044] Figure 2d This is a schematic diagram illustrating how public opinion keywords change throughout the entire time span in the embodiments of this application;

[0045] Figure 3 This is a flowchart of another keyword processing method in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the composition of the keyword processing device according to an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions in this application will now be clearly and completely described with reference to the accompanying drawings.

[0049] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present application and are not intended to limit the present application. Furthermore, the embodiments provided below are some embodiments for implementing the present application, and not all embodiments for implementing the present application. Unless otherwise specified, the technical solutions described in the present application can be implemented in any combination.

[0050] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of a processor, program, or software, etc.) in the method or apparatus that includes that element.

[0051] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0052] For example, the keyword processing method provided in this application includes a series of steps, but the keyword processing method provided in this application is not limited to the steps described. Similarly, the keyword processing apparatus provided in this application includes a series of modules, but the keyword processing apparatus provided in this application is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0053] This application can be implemented based on electronic devices, which may be thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, etc.

[0054] Terminal devices, servers, and other electronic devices can achieve corresponding functions through the execution of program modules. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc. They perform specific tasks or implement specific abstract data types. Computer systems can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0055] In some embodiments of this application, the keyword processing method can be implemented using a processor in a keyword processing device. The processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor.

[0056] Figure 1a This is a flowchart of a keyword processing method according to an embodiment of this application, such as... Figure 1a As shown, the process may include:

[0057] Step 100: Obtain the corpus of public opinion within a set time span; based on the corpus, determine multiple public opinion keywords.

[0058] For example, public opinion refers to sudden public opinion events, which are natural disasters, accidents, public health emergencies, or social security incidents that occur suddenly, cause or may cause serious harm, and require emergency response measures.

[0059] Here, the size of the time span can be set according to the actual situation, and this application embodiment does not limit it; for example, it can be the past three years, or other time periods.

[0060] In one implementation, the corpus may include each article related to the sudden public opinion event and the editing history of each article; here, there is no restriction on the source of each article, for example, it may be an article in Wikipedia, an article in a news website, etc.

[0061] In this embodiment of the application, after obtaining the corpus of public opinion within a set time span, the method of determining multiple public opinion keywords based on the corpus can be as follows: First, the corpus is preprocessed to obtain candidate keywords; then, the decay frequency and survival rate of each keyword in the candidate keywords are calculated to obtain the score of each keyword in the candidate keywords; finally, the keywords with higher rankings are determined as multiple public opinion keywords.

[0062] For example, preprocessing the corpus may include: removing low-quality edits from the corpus and then using a part-of-speech tagger to select nouns from the corpus as candidate keywords.

[0063] In the editing history of an article, attention should often be paid to edits that add or update content highly relevant to public opinion. However, many edits in the editing history fail to add or update key content, which will negatively impact the subsequent calculation of candidate keyword scores. Below are three types of low-quality edits that should be removed from the corpus: minor edits, bot edits, and malicious edits. Minor edits typically involve spelling corrections, grammar corrections, or formatting corrections that do not change the main content of the article; these edits will store almost identical text in the corpus. Bot edits refer to edits performed by machines, such as adding links or tags. Malicious edits include malicious deletions and offensive language.

[0064] For example, after deleting low-quality edits from the corpus, a segmenter (Chunker) can be used to segment the edited text of the article; then, part-of-speech tagging (POS Tagger) software can be used to assign part-of-speech tags, such as nouns, verbs, adjectives, etc., to each word in the segmented edited text; words containing noun part-of-speech tags are used as candidate keywords.

[0065] For example, in the editing history of an article, the editing of the article can be divided into long-term editing and short-term editing. Long-term editing often has rich information and is very stable throughout the editing history; they are generally rarely or never deleted. Short-term editing refers to those editing that has not been fully created, which may overlap with other editing content and will often be edited in a short period of time. When calculating the decay frequency (PHF) of each keyword in the candidate keywords, the candidate keywords in long-term editing need to be given a higher weight than those in short-term editing, as shown in formula (1):

[0066]

[0067] here, Let t(r) represent the word frequency of candidate keyword p in the k-th edit r, where ρ>0 is a decay factor.k ), t(r) k+1 ) represent the time points of the k-th and (k+1)-th edits of r, respectively; t(r1) and (r2) represent the time points of the k-th and (k+1)-th edits of r, respectively. n ) represent the time points of the 1st and nth edits of r, respectively; n represents the total number of edits. Where t(r) k ) and t(r k+1 The time span between ( ) is divided by the entire time span of the article, and is also considered as a weight to reduce the importance of short-term editing.

[0068] For example, in the article editing corpus S R In this context, if a candidate keyword p survives a long editing span, it indicates that candidate keyword p exhibits good quality; here, the survival rate SR of candidate keyword p is... (p) The calculation method is shown in formula (2).

[0069]

[0070] Here, scale(p) represents the total number of times p is edited from its first appearance to its last appearance, and contain(p) represents the total number of times p is edited including p from its first appearance to its last appearance.

[0071] For example, after obtaining the decay frequency and survival rate of each keyword in the candidate keywords according to the above formula, the score s(p) of each keyword in the candidate keywords is determined by formula (3). i ):

[0072]

[0073] Here, PF represents the i-th keyword p i The frequency of occurrence in n edits; ω represents the weight of the time feature.

[0074] For example, after obtaining the score of each keyword through formula (3), the scores of each keyword are arranged in descending order to obtain the sorting result.

[0075] For example, words ranking in the top 20% of the ranking results can be selected as public opinion keywords, or words ranking in the top 15% of the ranking results can be selected as public opinion keywords. This application embodiment does not limit this.

[0076] In this embodiment of the application, the above preprocessing of the acquired corpus can filter out some unimportant keywords and improve the detection effect of public opinion keywords.

[0077] Step 101: Determine the baseline time series corresponding to multiple public opinion keywords. Based on the baseline time series, determine the weight time series of each public opinion keyword among the multiple public opinion keywords. The baseline time series represents the time series corresponding to the total number of edits of multiple public opinion keywords.

[0078] For example, the time series corresponding to the total number of edits containing multiple keywords is used as the benchmark time series, and the public opinion outbreak time point in the benchmark time series is determined by the outbreak detection model, as shown in formula (4).

[0079] TS=[(s1,a1),...(s j ,a j (4)

[0080] Here, a j Let s represent the j-th time point in the baseline time series. j This represents the state at time point j, which corresponds to two cases: sudden or non-sudden; if s j If the state is sudden, then a j This is one of the points in time when public opinion suddenly broke out.

[0081] For example, the aforementioned outbreak detection model could be the Kleinberg outbreak detection model, which is a model for detecting meaningful outbreak structures in an article stream; it can be used to identify sudden public opinion events and form an overall hierarchical structure. If the frequency of a keyword increases rapidly, it can be defined as a "sudden outbreak." This model is used to identify sudden public opinion events in the article stream and generate status labels for these events.

[0082] In some embodiments, determining the weight time series of each public opinion keyword among multiple public opinion keywords based on a benchmark time series may include: performing component decomposition on the benchmark time series to obtain each component of the benchmark time series; the component represents a time series component including a single outbreak; and performing linear combination on each component of the benchmark time series to obtain the weight time series of each public opinion keyword.

[0083] For example, after determining the baseline time series corresponding to multiple public opinion keywords and the various public opinion outbreak time points in the series, the baseline time series can be decomposed based on the various public opinion outbreak time points to obtain the various component components of the baseline time series.

[0084] Here, each component of the base time series corresponds to a single burst of time series component; each time series component is a function of time t.

[0085] In some embodiments, performing component decomposition on a reference time series may include: performing component decomposition on the reference time series through a time series component decomposition model.

[0086] Exemplarily, there is no limitation on the type of the time series component decomposition model. For example, it can be a linear model, a multi-Gaussian model, or other types of models.

[0087] Exemplarily, Figure 1b is a schematic diagram of the linear model of the embodiments of the present application. As Figure 1b shown, the linear model can be defined by four line segments: when time t < a, cp(t) = 0. When time a < t < b, the line segment connects the points (a, 0) and (b, d). When time b < t < c, the line segment connects the points (b, d) and (c, residual). When t > c, cp(t) = residual. Here, a, b, and c respectively represent different time points; cp(t) represents the number of edits of each component at time t; the residual represents a value infinitely close to 0. See Figure 1b It can be seen that b is the time point of sudden public opinion, also known as the inflection point; the inflection point of the linear model can reflect how the change in the amount of editing causes the qualitative change in editing.

[0088] Exemplarily, the expression of the multi-Gaussian model is as shown in formula (5):

[0089]

[0090] Here, b is the time point of sudden public opinion, ρ is the decay factor, and f is the number of edits of each component at time t. Figure 1c is a schematic diagram of the multi-Gaussian model of the embodiments of the present application. See Figure 1c It can be seen that the multi-Gaussian model is often used to fit curves, can well describe sudden public opinion events, and can well reflect the long-tail trend.

[0091] In the embodiments of the present application, after performing component decomposition on the reference time series through a time series component decomposition model to obtain each component of the reference time series; then, by performing a linear combination of each component of the reference time series to fit the actual time series of each public opinion keyword.

[0092] Figures 1d to 1f is a schematic diagram of performing component decomposition on a time series through a linear model in the embodiments of the present application. As Figure 1d shown, it is the actual time series corresponding to a certain public opinion keyword; where the abscissa of the two-dimensional coordinate graph represents the time span in weeks, and the ordinate represents the number of edits. Here, by performing a linear combination of the Figure 1f three components, the fitted time series can be obtained, as Figure 1eThe solid line (the sum of the three components) is shown in the figure. (By...) Figures 1d to 1f It can be seen that a sharp rise in the curve in the time series can capture a sudden outbreak of a public opinion keyword, and a sudden outbreak can be represented by a component. The time corresponding to the peak point of each component is the sudden public opinion time point discovered by the above-mentioned sudden outbreak detection model.

[0093] For example, in the process of component decomposition of the baseline time series, since the fluctuations of the baseline time series are very complex, in order to reduce the complexity of component decomposition, only the sudden time series components including the time point of the public opinion outbreak are extracted here; that is, each component of the baseline time series includes the time series components of a single sudden outbreak.

[0094] In some embodiments, linearly combining the components of a reference time series may include: linearly combining the components of a reference time series according to a weight determination model; the weight determination model is used to determine the weights of the components of the reference time series.

[0095] In this application, before linearly combining the components of the baseline time series, it is necessary to determine the weights of each component of the baseline time series using a weight determination model. Then, based on the weights of each component, a linear combination of the components yields the weighted time series for each public opinion keyword.

[0096] In some embodiments, the training method of the weight determination model includes the following steps: inputting a public opinion keyword sample set into the weight determination model; processing the public opinion keyword sample set using the weight determination model to obtain the predicted weight of each keyword sample in the public opinion keyword sample set; and adjusting the network parameter values ​​of the weight determination model according to the predicted weight and labeled weight of each keyword sample in the public opinion keyword sample set.

[0097] Here, assume C = [c1,...,c M Let represent the components of a baseline time series, where parameter M is the dimension of the component decomposition, i.e., the number of component components. The value of the baseline time series C at time t, C(t), is defined as the value of each component c. m The sum of (t) is expressed as shown in formula (6):

[0098]

[0099] In order to find the components of the time series that best fit a given actual time series, a loss function for the weight determination model is defined, and its expression is shown in Equation (7):

[0100]

[0101] Here, G(t) is a given actual time series, assuming that the dimension M and time span T of each component in the time series are fixed. For the two time series component decomposition models mentioned above, a gradient descent-based loss minimization algorithm can be designed to determine the parameters of each time series component.

[0102] For example, the above C(t) can be used to represent any time series g(t); in this case, the time series component weights w = [w m ] m＝1,...,M It is an M-dimensional vector; the time series g(t) corresponding to the public opinion keywords is shown in formula (8):

[0103]

[0104] Here, a time series component weight w can be found through formula (9), and the weight w with the minimum loss can be found based on the gradient descent loss minimization algorithm.

[0105]

[0106] For example, the weight time series of each public opinion keyword can be represented by a weight vector on a time series component. To make the time series of various public opinion keywords comparable, a baseline time series can be component-decomposed, so that the weight time series of each public opinion keyword is represented as a linear combination of the baseline time series component decomposition (a weight vector on a baseline time series component). Thus, a baseline time series should include all important public opinion outbreak time points. In this embodiment, the total number of edits time series containing multiple public opinion keywords is used as the curve of the baseline time series component decomposition.

[0107] Figures 1g to 1i This is a schematic diagram illustrating the fitting of a baseline time series using a linear model according to an embodiment of this application, as shown below. Figure 1g As shown, this is the actual time series corresponding to the baseline time series; where the horizontal axis of this two-dimensional coordinate graph represents the time span in weeks, and the vertical axis represents the number of edits. Here, through the analysis of... Figure 1i The two component parts (corresponding to) Figure 1i By linearly combining the gray and black polylines in the graph, the fitted time series can be obtained, such as... Figure 1h The solid line in the diagram represents the sum of the two components. Figure 1h The time series C = C1 + C2; where C1 corresponds to Figure 1iThe component corresponding to the medium gray broken line; when time t < b1, C1(t) = 0; when time b1 < t < a1, the line segment connects the points (b1, 0) and (a1, d1); when time a1 < t < c1, the line segment connects the points (a1, d1) and (c1, residual); when t > c1, C1(t) = residual. C2 corresponds to Figure 1i The component corresponding to the medium black broken line; when time t < b2, C2(t) = 0; when time b2 < t < a2, the line segment connects the points (b2, 0) and (a2, d2); when time a2 < t < c2, the line segment connects the points (a2, d2) and (c2, residual); when t > c2, C2(t) residual. At this time, the loss function is as shown in formula (10):

[0108]

[0109] Here, the gradient descent method can be used to determine the parameters a1, b1, c1, d1, a2, b2, c2, d2, and residual in the loss function.

[0110] Step 102: Cluster multiple public opinion keywords according to the weight time series of each public opinion keyword to obtain the clustering results of multiple public opinion keywords.

[0111] In the embodiment of the present application, after obtaining the weight time series of each public opinion keyword among multiple public opinion keywords, cluster the multiple public opinion keywords according to the time series to obtain the clustering results of multiple public opinion keywords. The clustering results include multiple keyword clusters; it can be seen that through clustering, public opinion keywords with similar burst patterns can be clustered together to form each keyword cluster.

[0112] In some embodiments, clustering multiple public opinion keywords to obtain the clustering results of multiple public opinion keywords may include: determining the similarity between the weight time series of each public opinion keyword among multiple public opinion keywords; obtaining the clustering results of multiple public opinion keywords based on the similarity.

[0113] Here, there is no limitation on the implementation manner of clustering, which can be determined according to the actual situation. For example, the spherical k-means algorithm can be used for clustering, or other methods can be used for clustering.

[0114] For example, spherical k-means is a variant of k-means that uses cosine similarity as a similarity measure, ignores the magnitude of the vectors, and uses cluster cohesion as its maximizing objective function. Spherical k-means helps find keyword clusters with similar burst patterns. That is, if the spherical k-means algorithm is used for clustering, it achieves clustering by measuring the distance between the weight time series vectors of each public opinion keyword through cosine similarity. After clustering, the evolution of public opinion keywords and keyword clusters over time can be clearly understood.

[0115] Input: The weighted time series vector components w = {w'1, w'2, w'3, ..., w'} corresponding to each public opinion keyword. m} and the number of clusters K.

[0116] Output: The cluster identifier vector C = {c1, c2, ..., c3} i ,...,c n}, c i Given n keywords ∈{1,2,...,K}, W = {w1,w2,...,w...} n}Classification.

[0117] The expression for clustering using the spherical k-means algorithm is shown in formula (11):

[0118]

[0119] Where, p c (i) indicates that the i-th public opinion keyword w i Clustering to keyword cluster c i When ∈{1,2,...,K}, the keyword cluster c i The centroid vector ∈{1,2,...,K}, cos(w i ,p c (i) represents the i-th public opinion keyword w i The weighted time series vector and p c (i) Cosine similarity between them. The centroid vector of each keyword cluster is p1,...,pk; W={w1,w2,...,wk} n} is the keyword matrix for public opinion; P = {p1, p2, ..., pk} is the centroid vector matrix of the cluster.

[0120] In some embodiments, the above method may further include: after obtaining the clustering results of multiple public opinion keywords, visualizing the evolution process of the clustering results within a set time span.

[0121] In this embodiment, relevant public opinion keywords can be identified based on the cosine similarity of the weighted time series vectors on the baseline time series components. Here, each public opinion keyword is assigned to a horizontal line (Y = c).

[0122] The steps to visualize public opinion keywords are as follows:

[0123] Step 1. Based on the time point when the public opinion keyword first appeared, select a public opinion keyword p1, which has the earliest time point.

[0124] Step 2. Use the components of the weighted time series vector of p1 to plot p1. Use dark colors to represent high values ​​and light colors to represent low values. Align the components of the weighted time series vector according to their burst time points. In this way, the weighted time series vector of a public opinion keyword is plotted on a horizontal line.

[0125] Step 3. Select the sentiment keyword p2 with the highest cosine similarity to the weighted time series vector of p1. Plot the components of p2 as in Step 2, below the components of p1.

[0126] Step 4. Repeat Step 3 to plot the weight time series vectors of all public opinion keywords. If the cosine of the weight time series vectors of all remaining public opinion keywords and the weight time series vector of the current public opinion keyword is zero, then select a new public opinion keyword according to Step 1 (earliest time point).

[0127] Figure 2a This is a schematic diagram illustrating the various components of the base time series in the embodiments of this application, such as... Figure 2a As shown, the circles are the vertices of each time series component (component element), and each vertex represents a burst pattern. After representing the time series of public opinion keywords as a linear combination of the baseline time series components, a clustering method—spherical k-means—that measures the distance between the weighted time series vectors of the baseline time series component elements of the keywords through cosine similarity will be applied to cluster public opinion keywords with similar burst patterns. Figure 2b This is a schematic diagram illustrating the clustering of public opinion keywords into various keyword clusters in an embodiment of this application, such as... Figure 2b As shown, the keywords in each keyword cluster have similar burst patterns. Figure 2c The time series of representative keywords in each keyword cluster are displayed, and each representative keyword has a similar time series burst pattern to the public opinion keywords in the same cluster. Figure 2d It shows how public opinion keywords change and how their importance changes throughout the entire time span as events unfold and evolve.

[0128] This application proposes a method for analyzing the evolution of public opinion keywords based on the component decomposition of time series. Compared with the methods in related technologies that identify public opinion keywords through semantic methods such as Word2vec and TextRank, this application accurately extracts the characteristics of public opinion emergencies in the time series, visualizes the public opinion keywords at each public opinion emergencies time point, and effectively tracks the evolution process of public opinion keywords.

[0129] This application provides a keyword processing method, apparatus, electronic device, and computer storage medium. The method includes: acquiring corpus of public opinion within a set time span; determining multiple public opinion keywords based on the corpus; determining a baseline time series corresponding to the multiple public opinion keywords; and determining a weighted time series for each public opinion keyword based on the baseline time series. The baseline time series represents the time series corresponding to the total number of edits of the multiple public opinion keywords. The multiple public opinion keywords are clustered according to their weighted time series to obtain a clustering result. It can be seen that in this application embodiment, after obtaining multiple public opinion keywords from the corpus, the time series corresponding to the total number of edits of the multiple public opinion keywords is used as the baseline time series, and the weighted time series for each public opinion keyword is determined based on this baseline time series. Furthermore, clustering the multiple public opinion keywords using their weighted time series allows for the determination of the evolution pattern of the public opinion keywords from a temporal perspective, facilitating subsequent analysis and monitoring of public opinion events.

[0130] Figure 3 Here is a flowchart of another keyword processing method in the embodiments of this application, such as... Figure 3 As shown, the process may include:

[0131] Step A1: Obtain the data on public opinion within a set time span.

[0132] Step A2: Select multiple public opinion keywords from the corpus.

[0133] For example, data preprocessing can be performed by deleting low-quality edits and using part-of-speech tagging to select nouns to obtain candidate keywords in the corpus; the decay frequency and survival rate of the candidate keywords can be calculated, and the top 20% of the words can be selected as public opinion keywords.

[0134] Step A3: Time series component decomposition and evolution analysis of public opinion keywords.

[0135] For example, the time series containing the total number of edits of multiple public opinion keywords is used as the baseline time series. The Kleinberg burst detection model is used to detect bursts, identify the public opinion burst time points in the series, and decompose the series into components. The weight time series of each public opinion keyword is represented as a linear combination of the components of the baseline time series. Spherical k-means clustering is used to cluster public opinion keywords with similar burst patterns. The temporal evolution of public opinion keywords is visualized.

[0136] Figure 4 This is a schematic diagram of the composition structure of the keyword processing device according to an embodiment of this application, such as... Figure 3 As shown, the device includes: a first determining module 400, a second determining module 401, and a clustering module 402, wherein:

[0137] The first determining module 400 is used to acquire the corpus of public opinion within a set time span; based on the corpus, multiple public opinion keywords are determined;

[0138] The second determining module 401 is used to determine the baseline time series corresponding to multiple public opinion keywords, and based on the baseline time series, determine the weight time series of each public opinion keyword among the multiple public opinion keywords; the baseline time series represents the time series corresponding to the total number of edits of the multiple public opinion keywords;

[0139] Clustering module 402 is used to cluster multiple public opinion keywords based on the weight time series of each public opinion keyword, and obtain the clustering results of multiple public opinion keywords.

[0140] In some embodiments, the second determining module 401 is configured to determine the weight time series of each of a plurality of public opinion keywords based on a benchmark time series, including:

[0141] The baseline time series is decomposed into components to obtain the individual component components of the baseline time series; the component components represent time series components including single bursts.

[0142] By linearly combining the components of the baseline time series, the weighted time series of each public opinion keyword is obtained.

[0143] In some embodiments, the second determining module 401 is configured to perform component decomposition on the reference time series, including:

[0144] The baseline time series is decomposed using a time series component decomposition model.

[0145] In some embodiments, the second determining module 401 is configured to perform a linear combination of the components of the reference time series, including:

[0146] The weight determination model is used to linearly combine the components of the baseline time series; the weight determination model is used to determine the weights of each component of the baseline time series.

[0147] In some embodiments, the training method for the weight determination model includes the following steps:

[0148] Input the sample set of public opinion keywords into the weight determination model;

[0149] The weight determination model is used to process the public opinion keyword sample set to obtain the predicted weight of each keyword sample in the public opinion keyword sample set;

[0150] Based on the prediction weights and annotation weights of each keyword sample in the public opinion keyword sample set, the weights are adjusted to determine the network parameter values ​​of the model.

[0151] In some embodiments, the clustering module 402 is used to cluster multiple public opinion keywords to obtain clustering results for the multiple public opinion keywords, including:

[0152] Determine the similarity between the weighted time series of each of the multiple public opinion keywords;

[0153] Based on similarity, clustering results of multiple public opinion keywords were obtained.

[0154] In some embodiments, the clustering module 402 is further configured to:

[0155] After obtaining the clustering results of multiple public opinion keywords, the evolution of the clustering results over a set time span is visualized.

[0156] In practical applications, the first determining module 400, the second determining module 401, and the clustering module 402 can all be implemented by a processor located in an electronic device. The processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0157] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0158] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] Specifically, the computer program instructions corresponding to a keyword processing method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the computer program instructions corresponding to a keyword processing method in the storage media are read or executed by an electronic device, any of the keyword processing methods in the aforementioned embodiments are implemented.

[0160] Based on the same technical concept as the foregoing embodiments, see Figure 5 It illustrates an electronic device 500 provided in an embodiment of this application, which may include: a memory 501 and a processor 502; wherein,

[0161] Memory 501 is used to store computer programs and data;

[0162] The processor 502 is configured to execute a computer program stored in the memory to implement any of the keyword processing methods described in the foregoing embodiments.

[0163] In practical applications, the aforementioned memory 501 can be volatile memory, such as RAM; or non-volatile memory, such as ROM, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 502.

[0164] The processor 502 described above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understood that for different keyword processing devices, the electronic device used to implement the above processor function can also be other types, and this application embodiment does not specifically limit the specific types.

[0165] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0166] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0167] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0168] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0169] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0170] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0171] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0173] The above are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application.

Claims

1. A keyword processing method, characterized in that, The method includes: Acquire corpus of public opinion within a set time span; determine multiple public opinion keywords based on the corpus; A baseline time series corresponding to the plurality of public opinion keywords is determined, and based on the baseline time series, a weighted time series for each of the plurality of public opinion keywords is determined; the baseline time series represents the time series corresponding to the total number of edits of the plurality of public opinion keywords. Based on the weight time series of each public opinion keyword, the multiple public opinion keywords are clustered to obtain the clustering results of the multiple public opinion keywords; The step of determining the weighted time series of each of the plurality of public opinion keywords based on the benchmark time series includes: performing component decomposition on the benchmark time series to obtain each component of the benchmark time series; the component represents a time series component including a single outbreak; performing linear combination of each component of the benchmark time series according to a weight determination model to obtain the weighted time series of each public opinion keyword; the weight determination model is used to determine the weight of each component of the benchmark time series.

2. The method according to claim 1, characterized in that, Component decomposition of the reference time series includes: The baseline time series is decomposed into components using a time series component decomposition model.

3. The method according to claim 1, characterized in that, The training method for the weight determination model includes the following steps: Input the sample set of public opinion keywords into the weight determination model; The weight determination model is used to process the public opinion keyword sample set to obtain the predicted weight of each keyword sample in the public opinion keyword sample set; Based on the prediction weight and annotation weight of each keyword sample in the public opinion keyword sample set, the weights are adjusted to determine the network parameter values ​​of the model.

4. The method according to claim 1, characterized in that, The process of clustering the multiple public opinion keywords to obtain the clustering results includes: Determine the similarity between the weighted time series of each of the multiple public opinion keywords; Based on the similarity, the clustering results of the multiple public opinion keywords are obtained.

5. The method according to claim 1, characterized in that, The method further includes: After obtaining the clustering results of the multiple public opinion keywords, the evolution of the clustering results within a set time span is visualized.

6. A keyword processing device, characterized in that, The device includes: The first determining module is used to acquire corpus of public opinion within a set time span; and to determine multiple public opinion keywords based on the corpus. The second determining module is used to determine the baseline time series corresponding to the plurality of public opinion keywords, and based on the baseline time series, determine the weight time series of each of the plurality of public opinion keywords; the baseline time series represents the time series corresponding to the total number of edits of the plurality of public opinion keywords; The clustering module is used to cluster the multiple public opinion keywords according to the weight time series of each public opinion keyword, and obtain the clustering result of the multiple public opinion keywords; The second determining module is further configured to perform component decomposition on the benchmark time series to obtain each component of the benchmark time series; the component represents a time series component including a single burst; according to the weight determining model, the component of the benchmark time series is linearly combined to obtain the weighted time series of each public opinion keyword; the weight determining model is used to determine the weight of each component of the benchmark time series.

7. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method according to any one of claims 1 to 5.

8. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1 to 5.

Images