Text correction based topic modeling enhancement method and apparatus

Abstract

The application discloses a text correction-based topic modeling enhancement method and device, which comprises the following steps: extracting time distribution features of words, measuring the similarity between words according to the time distribution features, calculating the time expression ability index of the words, and obtaining a time special vocabulary set; extracting spatial distribution features of the words, measuring the spatial distribution difference of the words according to the spatial distribution features, generating a word distance matrix, performing vocabulary clustering based on the word distance matrix, and obtaining a spatial special vocabulary set; fusing point of interest data, extracting semantic distribution features of the words according to the point of interest data, calculating the semantic expression ability index of the words, and obtaining a semantic special vocabulary set; and modifying the text content based on the time special vocabulary set, the spatial special vocabulary set and the semantic special vocabulary set, to obtain an enhanced topic model; the application can extract spatiotemporal semantic special vocabularies and calculate expression ability indexes, and further correct the text to enhance topic modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geographic information, and in particular to a topic modeling enhancement method and device based on text correction. BACKGROUND

[0002] In the field of geographic big data mining, many scholars have studied information in time and space dimensions. With the development of social media, more and more geographic data carry text information. Text topic modeling is an effective means of utilizing text information. The input is a text set, each text is a set of words, and the output is a topic and a topic distribution of the text, wherein the topic is a distribution of words. Thus, the topic association of human activities in different time and space can be analyzed, which can effectively enhance the understanding of the connotation of human activities and play an important role in location-based services.

[0003] Although many scholars have studied this, the fusion of three-dimensional features of time, space and semantics still has problems. At present, many studies mainly use time and space information, ignoring semantic information, resulting in unclear activity topics. In the application process of the topic model, time and space semantic information is often only used for post-processing of the topic modeling result. In addition, existing studies ignore the deviation problem of the time semantic feature related to the data directly implied by the text, that is, the time point, space position and related semantics when the text is published are inconsistent with the time and space semantic information described by the text, resulting in unreliable topic modeling results.

[0004] Therefore, the prior art still needs to be improved. SUMMARY

[0005] The technical problem solved by the present application is that, in view of the defects of the prior art, the present application provides a topic modeling enhancement method and device based on text correction to solve the problem of inaccurate modeling results obtained by the existing text topic modeling method.

[0006] The technical solution adopted by the present application to solve the technical problem is as follows:

[0007] In a first aspect, the present application provides a topic modeling enhancement method based on text correction, comprising:

[0008] extracting time distribution features of words, measuring the similarity between words according to the time distribution features, calculating the time expression ability index of words, and obtaining a time special vocabulary set;

[0009] extracting spatial distribution features of words, measuring the spatial distribution difference of words according to the spatial distribution features, generating a word distance matrix, performing vocabulary clustering based on the word distance matrix, and obtaining a spatial special vocabulary set;

[0010] Fusing the interest point data, extracting semantic distribution features of words according to the interest point data, and calculating semantic expression ability indexes of the words to obtain a semantic special vocabulary set;

[0011] Modifying text content based on the time special vocabulary set, the space special vocabulary set, and the semantic special vocabulary set to obtain an enhanced topic model.

[0012] In an implementation manner, the time distribution features of the words are extracted, similarity between the words is measured according to the time distribution features, and time expression ability indexes of the words are calculated, including:

[0013] For each word, all data containing the word are extracted, time in the data is divided according to a preset time period, a time feature vector of the word is generated, the generated time feature vector is taken as a time distribution of the word, and a normalized time feature distribution of the word is obtained;

[0014] Differences between the time feature distribution of each word and time feature distributions corresponding to all texts are calculated, and a mean value of all differences is calculated, and words whose differences in the time feature distributions corresponding to all texts are greater than the mean value are taken as time special vocabularies;

[0015] A distribution element value set of all vocabularies in a specific time interval is obtained, a time distribution index of the word is calculated, and the time expression ability index of the word is determined according to the calculated time distribution index of the word.

[0016] In an implementation manner, the differences between the time feature distribution of each word and the time feature distributions corresponding to all texts are calculated, and the mean value of all differences is calculated, including:

[0017] The differences between the time feature distribution of each word and the time feature distributions corresponding to all texts are calculated based on cosine distances;

[0018] A sum of all differences is calculated, and the mean value of all differences is calculated according to the sum of differences.

[0019] In an implementation manner, the space distribution features of the words are extracted, space distribution differences of the words are measured according to the space distribution features, a word distance matrix is generated, and vocabulary clustering is performed based on the word distance matrix, including:

[0020] A space distribution of each word is obtained, and a space correlation parameter of each point in a space point set and an overall space point set is calculated in the space point set and the overall space point set;

[0021] A difference of the point in different sets is obtained according to the space correlation parameter, and a difference between the word and the overall space point set is aggregated to obtain a space expression ability index of the word.

[0022] Calculate the average difference of the spatial distribution, and the difference between the word and the whole spatial point set is greater than the average difference of the spatial distribution.

[0023] Calculate the spatial distance between the words, generate the word spatial distance matrix, and convert the distance matrix to two-dimensional coordinates of the words by dimension reduction and word clustering.

[0024] In an implementation, the difference between the points in different sets according to the spatial correlation parameters, and the difference between the word and the whole spatial point set after aggregation, includes:

[0025] The spatial correlation parameters of each point in the spatial point set and the whole spatial point set are taken as the difference of the spatial correlation, and the data volume and the positive and negative values are removed. After aggregation, the difference between the word and the whole spatial point set is obtained.

[0026] In an implementation, the fusion interest point data, the semantic distribution characteristics of the word are extracted according to the interest point data, and the semantic expression ability index of the word is calculated, including:

[0027] Get the interest point, and match the interest point according to the spatial relationship between the interest point and the data, and the spatial error of the data to obtain the semantic distribution of the word;

[0028] According to the main category characteristics in the semantic distribution, the number of main categories, the proportion of main categories and the entropy are calculated, and the semantic special word is determined according to the number of main categories, the proportion of main categories and the entropy.

[0029] Convert the semantic index to a probability value based on the normal assumption, and generate an integrated semantic index;

[0030] Calculate the expression ability of the word to a specific semantic category, and determine the semantic expression ability index according to the expression ability of the word to the specific semantic category.

[0031] In an implementation, the text content is modified based on the time special word set, the space special word set and the semantic special word set to obtain an enhanced topic model, including:

[0032] Label the time special word set, the space special word set and the semantic special word set, filter all words in each text in the original text set that are not in the label to obtain a filtered text set;

[0033] According to the category of the time special word set, the space special word set and the semantic special word set, the text words are labeled to obtain a text set based on the time-space semantic characteristics of the words;

[0034] Spatiotemporal semantic distribution vectors are generated based on the original text, and the names of the elements in the vectors are used as a new word set. A text set is then generated based on the set element values ​​of the new word set.

[0035] Based on the element values ​​in the distribution, different words in the new word set are copied multiple times to obtain a new text set;

[0036] The modified texts containing the new text set are applied to the topic model to obtain the enhanced topic model.

[0037] Secondly, the present invention provides a topic modeling enhancement device based on text correction, comprising:

[0038] The time distribution feature module is used to extract the time distribution features of words, measure the similarity between words based on the time distribution features, calculate the time expressive power index of words, and obtain a set of time-specific vocabulary.

[0039] The spatial distribution feature module is used to extract the spatial distribution features of words, measure the spatial distribution differences of words based on the spatial distribution features, generate a word distance matrix, and perform word clustering based on the word distance matrix to obtain a set of spatial special words.

[0040] The semantic distribution feature module is used to fuse interest point data, extract the semantic distribution features of words based on the interest point data, and calculate the semantic expressive power index of words to obtain a set of semantic special words.

[0041] The text correction module is used to modify the text content based on the set of time-specific words, the set of space-specific words, and the set of semantic-specific words to obtain an enhanced topic model.

[0042] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a text-based topic modeling enhancement program, and the text-based topic modeling enhancement program, when executed by the processor, is used to implement the operation of the text-based topic modeling enhancement method as described in the first aspect.

[0043] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium, storing a text-corrected topic modeling enhancement program, which, when executed by a processor, is used to implement the text-corrected topic modeling enhancement method as described in the first aspect.

[0044] The present invention, by employing the above technical solution, has the following effects:

[0045] This invention extracts spatiotemporal semantic special words, calculates the spatiotemporal semantic expressive power index of words to features, and then corrects the text based on spatiotemporal semantic special words while basically preserving the original text structure. This allows existing topic models to be directly applied to the corrected text, which can more effectively enhance the topic modeling results and solve the problem of inaccurate modeling results obtained by existing text topic modeling methods. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0047] Figure 1 This is a flowchart of a topic modeling enhancement method based on text correction in one implementation of the present invention.

[0048] Figure 2 This is a schematic diagram of the original and modified distribution of special vocabulary cases in one implementation of the present invention.

[0049] Figure 3 This is a schematic diagram comparing the effects of applying the original text and various modified texts to the LDA model in one implementation of the present invention.

[0050] Figure 4 This is a schematic diagram comparing the effects of the original text and various modified texts on different models in one implementation of the present invention.

[0051] Figure 5 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] Exemplary methods

[0055] Current research primarily utilizes spatiotemporal information while neglecting semantic information, resulting in unclear activity themes. Furthermore, in the application of topic models, spatiotemporal semantic information is often only used for post-processing of the topic modeling results. In addition, existing research overlooks the bias between the implicit spatiotemporal semantic information in the text and the temporal semantic features directly related to the data; that is, the time point, spatial location, and related semantics of the published text are inconsistent with the spatiotemporal semantic information described in the text, leading to unreliable topic modeling results.

[0056] To address the aforementioned issues, this invention provides a text-based topic modeling enhancement method. By extracting spatiotemporal semantic special words and calculating the spatiotemporal semantic expressive power index of the words to features, the text is then modified based on these spatiotemporal semantic special words while largely preserving the original text structure. This allows existing topic models to be directly applied to the modified text, more effectively enhancing topic modeling results and solving the problem of inaccurate modeling results obtained by existing text topic modeling methods.

[0057] like Figure 1 As shown, this embodiment of the invention provides a topic modeling enhancement method based on text correction, including the following steps:

[0058] Step S100: Extract the temporal distribution features of words, measure the similarity between words based on the temporal distribution features, calculate the temporal expressive power index of words, and obtain a set of time-specific vocabulary.

[0059] In this embodiment, the text-modification-based topic modeling enhancement method is applied to a terminal, which includes, but is not limited to, devices such as computers.

[0060] In this embodiment, firstly, the temporal distribution features of words are extracted, the similarity between words is measured, and the temporal expressive power index of words is calculated to obtain a set of temporal special words; then, the spatial distribution features of words are extracted, the spatial distribution differences are measured, a word distance matrix is ​​generated, and word clustering is performed to obtain a set of spatial special words; next, interest point data is fused, the semantic distribution features of words are extracted, and the semantic expressive power index of words is calculated to obtain a set of semantic special words; finally, the text content is modified based on the spatiotemporal semantic special words to enhance the topic model.

[0061] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0062] Step S101: For each word, extract all data containing each word, divide the time in the data according to the preset time period, generate the word's time feature vector, and use the generated time feature vector as the word's time distribution. After normalization, the word's time feature distribution is obtained.

[0063] In step S101, the word data is a text set, and the text is a set of words. j For one of the words, find all words containing that word. j The data records of the words are extracted, and their set is denoted as . Each text has a corresponding time t, which is mapped to three forms: hour, week, and date (i.e., the hour of the day, the day of the week, and the first day of the data), resulting in t = {hour, week, date}.

[0064] Preferably, the temporal feature vector TF of a word is obtained based on time t. j =[tf j,1 ,tf j,2 ,…,tf j,l ], tf∈{hour,week,date}; where l is the feature length, tf j,k For data sets The number of k in the middle; by normalizing the time feature vector. And the overall time distribution tf Res That is, to combine data sets Replace it with the entire text set ReS.

[0065] Step S102: Calculate the difference between the time feature distribution of each word and the time feature distribution of all texts, and calculate the mean of all differences. Words whose differences in the time feature distribution of all texts are greater than the mean are identified as time-specific words.

[0066] In step S102, firstly, the temporal feature distribution of each word and the temporal feature distribution corresponding to all texts (i.e., the overall temporal distribution tf) are calculated based on the cosine distance. ReS The difference between the differences is calculated; then, the sum of all differences is calculated, and the mean of all differences is calculated based on the sum of the differences.

[0067] Due to tf j With tf ReS Since both are vectors, the difference between them, temsif(tf), is calculated based on the cosine distance. j ,tf ReS ). temdif(tf) for all words j ,tf ReS Calculate the mean As a general level of difference, when temdif(tf j ,tf ReS When the value is greater than atd, move the word to the appropriate position. j As a special term related to time.

[0068] Step S103: Obtain the set of distribution element values ​​of all words within a specific time interval, calculate the time distribution index of the words, and determine the time expression ability index of the words based on the calculated time distribution index.

[0069] In step S103, the calculation of the word's time distribution index is specifically as follows:

[0070] For a time interval k, such as "hour 1" or "Monday", obtain the time interval k. k ={tf j,k |1≤j≤n word Treating it as a variable, modeling it based on a normal distribution, and calculating tdi. j,k =P(tf) k <tf j,k Further, only the time feature vectors of words are selected. j,k Elements greater than the mean. Then tdi j,k Multiply by temsif(tf) j ,tf ReS This serves as an index of the expressive power of word pairs over time interval k. Finally, the calculation... Size, remove words smaller than 50 j .

[0071] like Figure 1 As shown, in one implementation of this invention, the topic modeling enhancement method based on text correction further includes the following steps:

[0072] Step S200: Extract the spatial distribution features of words, measure the spatial distribution differences of words based on the spatial distribution features, generate a word distance matrix, and perform word clustering based on the word distance matrix to obtain a set of spatial special words.

[0073] In this embodiment, the main focus is on extracting the spatial distribution features of words, generating a word distance matrix by measuring the differences in spatial distribution, and then obtaining a set of spatially specific words through word clustering.

[0074] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0075] Step S201: Obtain the spatial distribution of each word, and calculate the spatial correlation parameters of each point in the spatial point set and the overall spatial point set.

[0076] In step S201, the word containing the word is first obtained. j All text ReS wordj The corresponding spatial point is used as a word. j The spatial distribution of the text, for one of the texts rei Spatial association SL can be achieved through re i Corresponding spatial location and Distance descriptions of other text spatial locations are expressed using formulas. calculate, Get the number of texts. Res to get re i Spatial relationships within the overall set of spatial points.

[0077] Step S202: Based on the spatial association parameters, the differences of points in different sets are obtained, and after aggregation, the differences between words and the overall spatial point set are obtained. The differences between words and the overall spatial point set are used as the spatial expressive power index of words.

[0078] In step S202, for the calculation of the difference between the word and the overall spatial point set, the spatial association parameter of each point in the spatial point set and the overall spatial point set is first used as the difference of spatial association, and the noise of data volume and positive and negative values ​​is removed. After aggregation, the difference between the word and the overall spatial point set is obtained.

[0079] In this embodiment, the text re i The differences between the two sets can be considered as differences in spatial correlation. Further removing the influence of data volume and positive / negative values, then... and ReS in re i Spatial differences are The overall difference between the two sets ReS and ReS is the mean difference of their commonalities.

[0080] Step S203: Calculate the average difference in spatial distribution, and identify words in the overall set of spatial points whose differences are greater than the average difference in spatial distribution as special spatial words.

[0081] In step S203, the average spatial distribution difference is And use it as an index of spatial expressive ability, if Greater than asd, word j These are special words in the space. Furthermore, the filtering scheme in S103 is used.

[0082] Step S204: Calculate the spatial distance between words, generate a word spatial distance matrix, and convert the distance matrix into two-dimensional coordinates of words by dimensionality reduction and perform word clustering.

[0083] In step S204, the distances between all spatial special words are calculated pairwise for each word. j1 With Word j2 The distance between corresponding points in two sets is used as the word distance. Finally, the word distance matrix can be obtained and reduced to two dimensions to simulate word coordinates for clustering. Labels are assigned to each word class. Furthermore, different labels are assigned to each noise word. j The identifier is denoted as cl(word) j1 ).

[0084] like Figure 1 As shown, in one implementation of this invention, the topic modeling enhancement method based on text correction further includes the following steps:

[0085] Step S300: Integrate the interest point data, extract the semantic distribution features of words based on the interest point data, and calculate the semantic expressive power index of words to obtain a set of semantic special words.

[0086] In this embodiment, in step S300, interest point data is fused, semantic distribution features of words are extracted, semantic expressive power index of words is calculated, and a set of semantic special words is obtained.

[0087] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0088] Step S301: Obtain interest points, and perform interest point matching based on the spatial relationship between interest points and data, as well as the spatial error of the data, to obtain the semantic distribution of words.

[0089] In step S301, for each text, according to the formula Find relevant points of interest, among which 100 represents the allowable error for the data. (The word...) j semantic distribution sd j yes Distribution of the number of different types of points of interest in China j ={vc j,1 ,vc j,2 ,…,vc j,e}, e is The number of different categories of interest points matched in Chinese text, VC j,t (1≤t≤e) refers to The number of texts that match the category t of the point of interest.

[0090] Step S302: Calculate the number of main categories, the proportion of main categories, and the entropy based on the main category features in the semantic distribution, and determine semantic special words based on the number of main categories, the proportion of main categories, and the entropy.

[0091] In step S302, the semantic distribution sd of each word is extracted. j The primary categories, i.e., those with a proportion greater than the mean, are calculated as the number of primary categories (ndt).j The proportion of major categories in PDT j , and the entropy of the main categories edt j As a semantic index, if ndt j and edt j It is less than the mean of the index corresponding to all words, while pdt j Greater than the mean, word j These are semantically special words. Furthermore, the filtering scheme in S103 is adopted.

[0092] Step S303: Based on the normal hypothesis, the semantic index is converted into a probability value and an integrated semantic index is generated.

[0093] In step S303, the main focus is on transforming the semantic indices and generating an integrated semantic index. Similar to the normal distribution modeling in S103, the three indices are transformed, including ndt. j =P(ndt) <ndt j ), pdt j =P(pdt) <pdt j ), edt j =P(edt) <edt j ), to obtain the integrated semantic index

[0094] Step S304: Calculate the expressive power of a word to a specific semantic category, and determine the semantic expressive power index based on the expressive power of the word to the specific semantic category.

[0095] Step S304 primarily involves calculating the expressive power of a word for a specific semantic category. (Word) j For type t1 The original directionality is Further, ISI j Multiply by rt j,t1 Get word j type t1 The index of expressive ability.

[0096] like Figure 1 As shown, in one implementation of this invention, the topic modeling enhancement method based on text correction further includes the following steps:

[0097] Step S400: Modify the text content based on the set of time-specific words, the set of space-specific words, and the set of semantic-specific words to obtain the enhanced topic model.

[0098] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0099] Step S401: Mark the set of time-specific words, the set of space-specific words, and the set of semantic-specific words; filter all words in each text of the original text set that are not marked to obtain a text set based on the filter.

[0100] In step S401, the text vocabulary is primarily filtered based on spatiotemporal semantic special terms. These special terms are denoted as SW, and the original text set is denoted as DOC. raw ={doc raw,1 ,doc raw,2 ,…,doc raw,nre Based on the filtering and modification of documents using SolidWorks (SW), a new document collection (DOC) is obtained. filter ={doc filter,1 ,doc filter,2 ,…,doc filter,nre}, where doc filter,i =[word j |wprd j ∈doc raw,i ∩SW].

[0101] Step S402: The text words are labeled according to the categories of the time-specific vocabulary set, the space-specific vocabulary set, and the semantic-specific vocabulary set to obtain a text set based on the spatiotemporal semantic characteristics of words.

[0102] In step S402, text words are labeled according to the categories of spatiotemporal semantic special words. Each word in the filtered document is labeled to indicate its spatiotemporal semantic characteristics; for example, the prefix "temsw-" is added to time-specific words, "spasw-" to space-specific words, and "semsw-" to semantic special words. If a word belongs to multiple special words, it is copied and the corresponding prefix is ​​added. This labeling process is denoted as tp, and the new document... doc cha,i =[tp(word) j )|word j ∈doc filter,i ].

[0103] Step S403: Generate a spatiotemporal semantic distribution vector based on the original text, and use the names of the elements in the vector as a new word set, and generate a text set based on the set element values ​​of the new word set.

[0104] In step S403, a new text is generated based on the spatiotemporal semantic distribution of the text. This is achieved by abstracting the document into a spatiotemporal semantic distribution vector, including the distribution of hours, weeks, dates, spaces, and semantics. Hours, weeks, and dates are derived from time divisions; spaces are all the identifiers after clustering; and semantics are the interest point categories. A new vocabulary set (DW) is obtained based on the name of each element in the vector. For example, after dividing by hours, there are 24 intervals from 0 to 23, and the first interval can be named "Hour 1". The element value in the vector is the sum of the expressive power indices of all words in the text for the feature corresponding to that element. Element names with values ​​greater than 0 are selected as vocabulary to form a new document set. doc dv,i =[dw j |value(dw j )>0], dw is a word in DW.

[0105] Step S404: Based on the element values ​​in the distribution, different words in the new word set are copied multiple times to obtain a new text set.

[0106] In step S404, the number of words in the text is adjusted based on the element values ​​in the distribution. This is done by expanding the element values ​​in the spatiotemporal semantic distribution vector and converting them to integers, then copying the corresponding element names according to their numerical values ​​to form a new document. doc dvwc,i =[repeateddw j |value(dw j )>0].

[0107] Step S405: Apply the various modified texts containing the new text set to the topic model to obtain the enhanced topic model.

[0108] In step S405, multiple revised texts are applied to the topic model to enhance its modeling results. By applying the topic model to multiple revised text sets, the topic distribution of the text and the word distribution of the topics are obtained. The feature distribution corresponding to the topics can be obtained by multiplying the word's expressive power index by the probability of the word in the topic, and then summing all words.

[0109] To illustrate the advantages of the topic modeling enhancement method provided in this embodiment of the invention over the original distribution used in other existing methods, the results of extracting special words in spatiotemporal semantics and the effect of enhancing the topic model are demonstrated.

[0110] like Figure 2 As shown, Figure 2The text presents the original and revised distributions of the word "morning." The original distribution is commonly used in other studies, while the revised distribution is calculated based on the representativeness index. It can be seen that the original distribution peaks at 8:00 AM, and its values ​​are non-zero outside of the morning time period, which is inconsistent with the meaning of the word. This indicates a discrepancy between the published time of the text and the time stated in the text content. The revised distribution effectively resolves this issue.

[0111] like Figure 3 As shown, Figure 3 The image shows the effect of applying LDA with 50 topics to different texts, including DOCs commonly used in document-related fields. raw The original and modified vocabulary distributions (labeled "Raw1" and "Raw2" respectively) were used, based on the modified document DOC. filter DOC cha DOC dv With DOC dvwc The results were denoted as Filter, Cha, Dv, and Dvwc, respectively. Additionally, a weighted version of TF-IDF was tested. It was found that Raw1 and Raw1 TF-IDF performed the worst. Ignoring probability values, Dv and Dv TF-IDF also performed poorly. However, with probability added, Dvwc and Dvwc TF-IDF achieved the best performance. The topic modeling enhancement method provided in this invention can improve the modeling effect of human activity topics by extracting spatiotemporal semantic special words to correct documents, which is superior to the method used in existing studies that directly uses the original uncorrected spatiotemporal distribution.

[0112] like Figure 4 As shown, Figure 4 The text demonstrates the effects of different corrected or original texts on different topic models. It shows that corrections based on different corrected documents generally yield better results than uncorrected documents. While Dvwc and its TF-IDF version are mostly used for physical tests (PT), Cha can be used for PT.

[0113] In summary, the topic modeling enhancement method provided by the embodiments of the present invention has advantages over other mainstream methods.

[0114] This embodiment achieves the following technical effects through the above technical solution:

[0115] This embodiment extracts spatiotemporal semantic special words, calculates the spatiotemporal semantic expressive power index of words to features, and then corrects the text based on spatiotemporal semantic special words while basically preserving the original text structure. This allows existing topic models to be directly applied to the corrected text, which can more effectively enhance the topic modeling results and solve the problem of inaccurate modeling results obtained by existing text topic modeling methods.

[0116] Exemplary device

[0117] Based on the above embodiments, the present invention also provides a topic modeling enhancement device based on text correction, comprising:

[0118] The time distribution feature module is used to extract the time distribution features of words, measure the similarity between words based on the time distribution features, calculate the time expressive power index of words, and obtain a set of time-specific vocabulary.

[0119] The spatial distribution feature module is used to extract the spatial distribution features of words, measure the spatial distribution differences of words based on the spatial distribution features, generate a word distance matrix, and perform word clustering based on the word distance matrix to obtain a set of spatial special words.

[0120] The semantic distribution feature module is used to fuse interest point data, extract the semantic distribution features of words based on the interest point data, and calculate the semantic expressive power index of words to obtain a set of semantic special words.

[0121] The text correction module is used to modify the text content based on the set of time-specific words, the set of space-specific words, and the set of semantic-specific words to obtain an enhanced topic model.

[0122] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown.

[0123] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or mobile terminal.

[0124] When executed by the processor, this computer program is used to implement the text-modification-based topic modeling enhancement method.

[0125] It will be understood by those skilled in the art that Figure 5 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a text-based topic modeling enhancement program, which, when executed by the processor, is used to implement the operations of the text-based topic modeling enhancement method as described above.

[0127] In one embodiment, a storage medium is provided, wherein the storage medium stores a text-based topic modeling enhancement program, which, when executed by a processor, is used to implement the operations of the text-based topic modeling enhancement method described above.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.

[0129] In summary, this invention provides a method and apparatus for enhancing topic modeling based on text correction. The method includes: extracting temporal distribution features of words, measuring the similarity between words based on the temporal distribution features, calculating the temporal expressive power index of words, and obtaining a set of temporal special words; extracting spatial distribution features of words, measuring the spatial distribution differences of words based on the spatial distribution features, generating a word distance matrix, and performing word clustering based on the word distance matrix to obtain a set of spatial special words; fusing interest point data, extracting semantic distribution features of words based on the interest point data, and calculating the semantic expressive power index of words to obtain a set of semantic special words; modifying text content based on the set of temporal special words, the set of spatial special words, and the set of semantic special words to obtain an enhanced topic model. This invention can extract spatiotemporal semantic special words and calculate expressive power indices to further correct text and enhance topic modeling.

[0130] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A topic modeling enhancement method based on text correction, characterized in that, include: Extract the temporal distribution features of words, measure the similarity between words based on the temporal distribution features, calculate the temporal expressive power index of words, and obtain a set of time-specific vocabulary; Extract the spatial distribution features of words, measure the spatial distribution differences of words based on the spatial distribution features, generate a word distance matrix, and perform word clustering based on the word distance matrix to obtain a set of spatially specific words; By integrating interest point data, semantic distribution features of words are extracted based on the interest point data, and the semantic expressive power index of words is calculated to obtain a set of semantically special words; Based on the set of time-specific vocabulary, the set of space-specific vocabulary, and the set of semantic-specific vocabulary, the text content is modified to obtain an enhanced topic model; The process of modifying text content based on the set of temporal special words, the set of spatial special words, and the set of semantic special words to obtain an enhanced topic model includes: The time-specific vocabulary set, the space-specific vocabulary set, and the semantic-specific vocabulary set are marked, and all words in each text of the original text set that are not marked are filtered to obtain a text set based on the filter. Text words are labeled according to the categories of the time-specific vocabulary set, the space-specific vocabulary set, and the semantic-specific vocabulary set to obtain a text set based on the spatiotemporal semantic characteristics of words; Spatiotemporal semantic distribution vectors are generated based on the original text, and the names of the elements in the vectors are used as a new word set. A text set is then generated based on the set element values ​​of the new word set. Based on the element values ​​in the distribution, different words in the new word set are copied multiple times to obtain a new text set; The modified texts containing the new text set are applied to the topic model to obtain the enhanced topic model.

2. The topic modeling enhancement method based on text correction according to claim 1, characterized in that, The extracted time distribution features of words are used to measure the similarity between words and calculate the time expressive power index of words, including: For each word, extract all data containing each word, divide the time in the data according to a preset time period, generate the word's time feature vector, and use the generated time feature vector as the word's time distribution. After normalization, the word's time feature distribution is obtained. Calculate the difference between the temporal feature distribution of each word and the temporal feature distribution of all texts, and calculate the mean of all differences. Words whose differences in the temporal feature distribution of all texts are greater than the mean are identified as temporal special words. Obtain the set of distribution element values ​​of all words within a specific time interval, calculate the time distribution index of the words, and determine the time expressive power index of the words based on the calculated time distribution index.

3. The topic modeling enhancement method based on text correction according to claim 2, characterized in that, The calculation of the difference between the temporal feature distribution of each word and the temporal feature distribution of all texts, and the calculation of the mean of all differences, includes: The difference between the temporal feature distribution of each word and the temporal feature distribution of all texts is calculated based on cosine distance. Calculate the sum of all differences, and then calculate the mean of all differences based on the sum of the differences.

4. The topic modeling enhancement method based on text correction according to claim 1, characterized in that, The process involves extracting spatial distribution features of words, measuring spatial distribution differences of words based on these features, generating a word distance matrix, and performing vocabulary clustering based on the word distance matrix, including: Obtain the spatial distribution of each word, and calculate the spatial association parameters of each point in the spatial point set and the overall spatial point set. The differences between points in different sets are obtained based on the spatial association parameters. After aggregation, the differences between words and the overall spatial point set are obtained. The differences between words and the overall spatial point set are used as the spatial expressive power index of words. Calculate the average difference in spatial distribution, and identify words in the overall set of spatial points whose differences are greater than the average difference in spatial distribution as special spatial words. Calculate the spatial distance between words, generate a word spatial distance matrix, and convert the distance matrix into two-dimensional coordinates of words by dimensionality reduction and then perform word clustering.

5. The topic modeling enhancement method based on text correction according to claim 4, characterized in that, The step of obtaining the differences between points in different sets based on the spatial association parameters, and then aggregating them to obtain the differences between the words and the overall spatial point set, includes: The spatial association parameter of each point in the spatial point set and the overall spatial point set is used as the difference in spatial association. After removing the noise of the data volume and positive and negative values, the difference between the word and the overall spatial point set is obtained after aggregation.

6. The topic modeling enhancement method based on text correction according to claim 1, characterized in that, The fused interest point data, based on which semantic distribution features of words are extracted and semantic expressive power indices of words are calculated, includes: Identify points of interest and perform point of interest matching based on the spatial relationship between the points of interest and the data, as well as the spatial error of the data, to obtain the semantic distribution of words; Based on the main category features in the semantic distribution, calculate the number of main categories, the proportion of main categories, and the entropy; and determine semantic special words based on the number of main categories, the proportion of main categories, and the entropy. Based on the normality assumption, semantic indices are converted into probability values, and ensemble semantic indices are generated. Calculate the expressive power of a word to a specific semantic category, and determine a semantic expressive power index based on the expressive power of the word to the specific semantic category.

7. A text-modification-based topic modeling enhancement apparatus, used to implement the text-modification-based topic modeling enhancement method as described in any one of claims 1-6, characterized in that, include: The time distribution feature module is used to extract the time distribution features of words, measure the similarity between words based on the time distribution features, calculate the time expressive power index of words, and obtain a set of time-specific vocabulary. The spatial distribution feature module is used to extract the spatial distribution features of words, measure the spatial distribution differences of words based on the spatial distribution features, generate a word distance matrix, and perform word clustering based on the word distance matrix to obtain a set of spatial special words. The semantic distribution feature module is used to fuse interest point data, extract the semantic distribution features of words based on the interest point data, and calculate the semantic expressive power index of words to obtain a set of semantic special words. The text correction module is used to modify the text content based on the set of time-specific words, the set of space-specific words, and the set of semantic-specific words to obtain an enhanced topic model.

8. A terminal, characterized in that, include: The processor and memory, the memory storing a text-based topic modeling enhancement program, which, when executed by the processor, is used to implement the operation of the text-based topic modeling enhancement method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a text-based topic modeling enhancement program, which, when executed by a processor, is used to implement the operation of the text-based topic modeling enhancement method as described in any one of claims 1-6.

Images