Text processing method, server, and storage medium

Abstract

The application discloses a text processing method, comprising: collecting original text, and performing data preprocessing on the original text to obtain a list of candidate words; obtaining an anchor word in response to a selection input of a user according to the list of candidate words; processing a preset initial model according to the anchor word to obtain a theme word extraction model corresponding to the original text; and extracting theme words from the original text according to the theme word extraction model to obtain the theme words. The text processing method can preprocess the original text to obtain the list of candidate words, further generate the anchor word according to the selection of the user in the list of candidate words, and then extract the theme words from the original text according to the theme word extraction model, so that the text information is effectively screened with a purpose by referring to the anchor word given by the user.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method for extracting text topic content, a server, and a storage medium. Background Technology

[0002] Current technologies typically employ pre-defined extraction models to extract keywords from text based on pre-configured parameters. They rarely consider user needs when selecting extracted keywords, or they often directly filter the entire text to provide anchor words for user selection. These methods result in keywords with low relevance to the topic, and the anchor word selection process is cumbersome. Therefore, developing an efficient text processing method that meets user needs is a pressing issue. Summary of the Invention

[0003] This application provides a text processing method, a server, and a storage medium.

[0004] This application provides a text processing method, including:

[0005] Collect the raw text and perform data preprocessing on the raw text to obtain a list of candidate words;

[0006] In response to the user's input based on the selection of the candidate word list, the anchor word is obtained;

[0007] Based on the preset initial model for anchor word processing, a topic word extraction model corresponding to the original text is obtained;

[0008] The original text is used to extract keywords based on the keyword extraction model, thus obtaining keyword terms.

[0009] Thus, the text processing method of this application can preprocess the original text to obtain a list of candidate words, and further generate anchor words based on the user's selection in the list of candidate words. Then, based on the topic word extraction model, topic words are extracted from the original text, effectively filtering the text information by referring to the anchor words given by the user.

[0010] In some implementations, the step of collecting raw text and performing data preprocessing on the raw text to obtain a list of candidate words includes:

[0011] The text title of the original text is segmented into words;

[0012] The segmented data is then filtered.

[0013] The candidate words are obtained from the filtered data to form a candidate word list.

[0014] Thus, by performing word segmentation and filtering on the text title, the detection range of candidate words can be quickly narrowed, a candidate word list can be generated, and the processing efficiency of the text processing method can be improved.

[0015] In some implementations, the filtering process on the segmented data includes:

[0016] The candidate words are obtained by filtering out stop words and invalid words from the segmented data according to the preset list of prohibited words, and the list of prohibited words includes the preset stop words and invalid words.

[0017] Thus, the data obtained from word segmentation is processed according to the pre-set list of prohibited words to filter out stop words and invalid words contained in the initially obtained text titles, generate candidate words with higher data validity, and then generate a candidate word list to ensure the validity of the obtained candidate words.

[0018] In some embodiments, the filtering process on the segmented data further includes:

[0019] The data after filtering is subjected to the first deduplication process to remove identical word segments.

[0020] Thus, the data after filtering out stop words and invalid words undergoes the first deduplication process, thereby removing completely identical word segments from the data and improving the effectiveness of the candidate words.

[0021] In some embodiments, the filtering process on the segmented data further includes:

[0022] The data after the first removal process is subjected to a second deduplication process to filter out words with the same semantic meaning from the word segmentation, thereby obtaining the candidate words.

[0023] Thus, the data after the first deduplication process is subjected to a second deduplication process based on semantics, which removes synonyms and improves the effectiveness of the candidate words.

[0024] In some implementations, the step of processing the anchor words according to a preset initial model to obtain a topic word extraction model corresponding to the original text includes:

[0025] Obtain the preset initial model;

[0026] The initial model is configured based on the anchor weight value and the anchor word to obtain the topic word extraction model.

[0027] Thus, based on the anchor words and anchor weight values ​​selected by the user, the pre-set initial model is configured to obtain model data that matches the original text. Users can control the correlation between the selected topic words and anchor words according to the anchor weight values, thereby obtaining a topic extraction model that can better fit the user's needs.

[0028] In some implementations, the step of extracting keywords from the original text based on the keyword extraction model to obtain keyword terms includes:

[0029] Perform word segmentation on the document body of the original text;

[0030] The data after the word segmentation operation is filtered out according to the preset list of prohibited words to obtain the words to be processed;

[0031] The topic words are obtained by extracting topic words from the words to be processed according to the topic word extraction model.

[0032] Thus, by performing relevant preprocessing on the original document text, the effectiveness of the words to be processed is improved, and the efficiency of subject term extraction is also increased.

[0033] In some implementations, the step of extracting topic terms from the words to be processed according to the topic term extraction model to obtain the topic terms includes:

[0034] The topic relevance between each word to be processed in the document and the anchor word is calculated based on the topic word extraction model.

[0035] The words to be processed are sorted in descending order of their topic relevance scores.

[0036] Based on the anchor point weight value, a predetermined number of words to be processed are extracted according to the sorting order of the words to be processed to obtain the topic words.

[0037] In this way, the topic words can be extracted based on the relevance between the words to be processed and the anchor words.

[0038] This application also provides a server, including:

[0039] The processor and a memory communicatively connected to the processor, wherein the memory stores instructions that the processor can implement the above-described text processing method by executing the instructions.

[0040] This application also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the above-described text processing method.

[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0042] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description taken in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 This is a flowchart illustrating a text processing method provided in an embodiment of this application.

[0044] Figure 2 This is a flowchart illustrating a specific text processing method provided in an embodiment of this application.

[0045] Figure 3 This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0046] Figure 4 This is a schematic diagram of a word segmentation model provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram of a specific server provided in an embodiment of this application.

[0048] Figure 6 This is a schematic diagram of the candidate word list provided in an embodiment of this application.

[0049] Figure 7 This is a schematic diagram of a word cloud for a subject matter embodiment of this application.

[0050] Figure 8 A schematic diagram illustrating the list of keywords provided for embodiments of this application. Detailed Implementation

[0051] The present application is described in detail below, with examples shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The following description with reference to the accompanying drawings is exemplary and intended to explain the present application, and should not be construed as limiting the present application.

[0052] Please see Figure 1 and 2 This application provides a text processing method, including:

[0053] S10: Collect the raw text and perform data preprocessing on the raw text to obtain a list of candidate words;

[0054] S20: In response to the user's input based on the selection of the candidate word list, obtain the anchor word;

[0055] S30: Based on the anchor word processing, a preset initial model is obtained to obtain the corresponding original text's topic word extraction model;

[0056] S40: Extract keywords from the original text using the keyword extraction model to obtain keyword terms.

[0057] This application also provides a server, which can implement the text processing method of this application. The server includes a memory and a processor. The memory stores a computer program, and the processor is used to acquire raw text, perform data preprocessing on the raw text to obtain a list of candidate words, and obtain anchor words in response to user input based on the selection of candidate words from the list; process a preset initial model based on the anchor words to obtain a topic word extraction model corresponding to the raw text; and extract topic words from the raw text based on the topic word extraction model to obtain topic words.

[0058] Specifically, the original text is the target text from which the user wishes to extract keywords, which can be patents, news articles, academic papers, etc. Keywords can be considered as key terms in the text. Anchor terms represent the user's keyword extraction preferences; understandably, the extracted keywords revolve around the anchor terms, or in other words, the extracted keywords are related to the anchor terms.

[0059] A text can be viewed as consisting of several words. After preprocessing the original text, a list of candidate words is obtained. Users can select words of interest from this list as anchor words, and then use a topic word extraction model to extract topic words associated with the anchor words from the text. In this application, the number of original texts is not limited; there can be one or more. The original text is preprocessed using a relevant model. The list of candidate words can be obtained by processing multiple original texts.

[0060] To improve processing efficiency, preprocessing of the original text can be performed on content at predetermined structural points within the text. For example, for news articles, this could be the headline; for patents, it could be the headline and / or abstract; and for academic papers, it could be the headline and / or abstract.

[0061] The server can be pre-configured with a model for extracting keywords. It can also use the anchor words selected by the user as prior knowledge to configure the initial model, thereby obtaining a keyword extraction model that can filter words related to the anchor words and apply it to the original text. Then, the model can be used to extract keywords from the original text and generate keywords that meet the user's needs.

[0062] Thus, the text processing method of this application can preprocess the original text to obtain a list of candidate words, and further generate anchor words based on the user's selection in the list of candidate words. Then, based on the topic word extraction model, topic words are extracted from the original text, effectively filtering the text information by referring to the anchor words given by the user.

[0063] In some implementations, S10 includes the following steps:

[0064] S101: Perform word segmentation on the text title of the original text;

[0065] S102: Filter the data after word segmentation;

[0066] S103: Based on the filtered data, a list of candidate words is formed.

[0067] In some implementations, the processor is used to perform word segmentation on the text title of the original text, to filter the segmented data, and to generate a candidate word list based on the filtered data.

[0068] Understandably, a text's title is generally a high-level summary of the entire text, and key keywords often appear in the title. The title in this application can be either the overall title of the entire text or a subheading within the text; no limitation is made here.

[0069] In practical processing, relevant models can be used to extract the title of the text and use the text title as the source for selecting candidate words.

[0070] Specifically, the text title can be segmented and filtered. Longer text titles can be divided into words, and words that are not meaningful, the same, or semantically similar can be filtered out to obtain candidate words. Then, a candidate word list can be formed based on the candidate words.

[0071] Thus, by performing word segmentation and filtering on the text title, the detection range of candidate words can be quickly narrowed, a candidate word list can be generated, and the processing efficiency of the text processing method can be improved.

[0072] In some implementations, S103 includes:

[0073] Stop words and invalid words in the segmented data are filtered out based on a preset list of prohibited words to obtain candidate words and form a candidate word list.

[0074] In some implementations, the processor is used to filter out stop words and invalid words in the segmented data according to a preset list of prohibited words, so as to obtain candidate words to form a candidate word list.

[0075] Specifically, in this application, the segmented data is filtered based on a pre-defined list of prohibited words. The list of prohibited words can be pre-set or updated as needed. By using the list of prohibited words, stop words and invalid words in the initially obtained text titles can be filtered out to generate candidate words with higher data validity, thereby generating a candidate word list for generating anchor words.

[0076] Specifically, the stopped words in the prohibited word list can be illegal words or words irrelevant to the user's needs. Invalid words can be words with low relevance to the topic, such as conjunctions, verbs, adjectives, etc. Understandably, these words usually serve a modifying and connecting function rather than actually referring to specific actions or things, and cannot reflect the core content of the title or text.

[0077] Thus, the data obtained from word segmentation is processed according to the pre-set list of prohibited words to filter out stop words and invalid words contained in the initially obtained text titles, generate candidate words with higher data validity, and then generate a candidate word list to ensure the validity of the obtained candidate words.

[0078] In some embodiments, S103 further includes:

[0079] The data after filtering is subjected to the first deduplication process to remove identical word segments.

[0080] In some factual approaches, the processor is used to perform a first deduplication process on the filtered data to remove identical word segments.

[0081] It's understandable that different text titles in multiple original texts might contain the same word. By selecting only one of these identical words as candidate words, the other identical words can be filtered out, thus improving the effectiveness of the candidate words.

[0082] Thus, the data after filtering out stop words and invalid words undergoes the first deduplication process, thereby removing completely identical word segments from the data and improving the effectiveness of the candidate words.

[0083] In some embodiments, S103 further includes:

[0084] The data after the first removal process is subjected to a second deduplication process to filter out words with the same semantic meaning from the word segmentation, thereby obtaining candidate words.

[0085] In some implementations, the processor is used to perform a second deduplication process on the data after the first deduplication process to filter out words with the same semantic meaning from the word segmentation, thereby obtaining candidate words.

[0086] Specifically, for multiple texts, there may be multiple semantically similar ways of describing the same object; that is, although the forms of expression are different, their meanings are the same. To improve the effectiveness of candidate words and to facilitate users in selecting anchor words, one of the most frequently occurring words can be retained as a candidate word, while other synonyms are filtered out. After the above word segmentation, filtering, and multiple deduplication processes, the final candidate words can be obtained.

[0087] Thus, the data after the first deduplication process is subjected to a second deduplication process based on semantics, which removes synonyms and improves the effectiveness of the candidate words.

[0088] In some implementations, S30 includes:

[0089] S301: Obtain the preset initial model;

[0090] S302: Based on the anchor weight value and anchor word configuration, the initial model is obtained to obtain the topic word extraction model.

[0091] In some implementations, the processor is used to obtain a preset initial model and to configure the initial model according to the anchor weight value and anchor words to obtain a topic word extraction model.

[0092] Specifically, this application can obtain specific models for different original texts according to user needs. First, a preset initial model is obtained, and the anchor words selected by the user are input into the initial model as prior knowledge, thereby obtaining a topic word extraction model applicable to this topic word extraction. Users can also set anchor weight values ​​to configure the topic word extraction model. The anchor weight values ​​can be used to configure the degree of correlation between the extracted topic words and anchor words, thereby controlling the correlation between the selected topic words and anchor words, and thus obtaining a topic word extraction model that can better fit the user's needs.

[0093] Specifically, the topic word extraction model used in this application can be a CorEx (Correlation Explanation) semi-supervised topic word extraction model based on prior knowledge. Through this model, the degree of association between each word in the original text and the anchor word can be calculated, and each word can be identified as a topic word according to the specific value of the degree of association.

[0094] Specifically, the semi-supervised topic extraction model makes no structural prior assumptions about the generation of the data. Instead, similar to information gain, it uses the difference in total correlation (TC) to find the topic words most relevant to the anchor words. The entropy of the random variable X (the set of all words in the original text) is: H(X) = -∑ Xp(x)log(p(x)). The mutual information between the two random variables is: I(X1:X2)=H(X1)+H(X2)-H(X1,X2), where TC, which characterizes the degree of association between each word and the topic word, is defined as: Where G is a subset of X, and TC describes the degree of dependency between variables in the set. When G contains only two variables, TC is equivalent to the mutual information between the two variables. The conditional TC is defined as: TC(X|Y)=∑ i H(X i |Y)-H(X|Y). Here, Y is the final keyword. Therefore, we can use the difference between TC and conditional TC to measure the impact of a keyword Y on the correlation between variables in variable X. The mutual information of X and Y is represented as follows:

[0095] A better keyword should be a representation of a document, and the document's TC value should be maximized. Therefore, we need to find the keyword X corresponding to the maximum value of TC(X:Y), which is the keyword with the highest fit to Y. Using this model, this method can calculate... Where p(y|x) represents the probability that each document x belongs to a certain topic y.

[0096] The final optimization function is expressed as follows: maxβI(z:y j )-I(x:y j ). Where β is the anchor weight value, used to control the influence of anchor word z on the extraction of topic word y. Therefore, the core algorithm of this model lies in iterating the following three steps: p t (y j )=∑ x p t (y j |x)p(x), p t (x i |y j )=∑ x p t (y j |x)p(x) / p t (y j ), Where t represents the t-th iteration. This model defines the document topic using p(Y|X), and further updates the probability p(y|X) of each topic according to the formula. j ), topic distribution p(y) for each word i |x i ), and the topic distribution p(y) of each document. j|x). Here, p(Y|X) is randomly set during initialization when the initial model is called to achieve the aforementioned iterative process. Then, based on the iterative calculation process, the correlation between each word in the original text and the user-defined anchor words is obtained.

[0097] The CorEx semi-supervised topic extraction model can be understood as performing topic clustering on words in a text, while introducing the variable α during the modeling process. i,j During optimization, certain values ​​can be fixed, i.e., some anchor words can be set in advance. After optimization, these words can be clustered together with other related words. The clustering process, as shown above, mainly involves calculating entropy values ​​and using entropy values ​​to describe the objective function.

[0098] Thus, based on the anchor words and anchor weight values ​​selected by the user, the pre-set initial model is configured to obtain model data that matches the original text. Users can control the correlation between the selected topic words and anchor words according to the anchor weight values, thereby obtaining a topic extraction model that can better fit the user's needs.

[0099] In some implementations, S40 includes:

[0100] S401: Perform word segmentation on the original document text;

[0101] S402: Based on the preset list of prohibited words, perform a filtering operation on the data after word segmentation to obtain the words to be processed;

[0102] S403: Extract keywords from the words to be processed using the keyword extraction model to obtain keywords.

[0103] In some implementations, the processor is used to perform word segmentation on the document body of the original text, and to perform filtering operations on the segmented data according to a preset list of prohibited words to obtain words to be processed, and to extract keywords from the words to be processed according to a keyword extraction model to obtain keywords.

[0104] Specifically, this application utilizes an anchor word and subject word extraction model selected from candidate words generated through the title of this document to extract subject words from the main body of the original text, thereby extracting words related to the anchor words in the main body. Similar to processing the text title, to improve the efficiency of subject word extraction, word segmentation, filtering, and deduplication can also be performed on the main body of the text to obtain words to be processed, and then subject words can be extracted from these words. The models and methods used for word segmentation, filtering, and deduplication of the main body of the text are basically the same as those used for processing the text title. It should be noted that since the extraction is of subject words, only duplicate word deduplication is required for the main body, and deduplication of synonyms is not necessary.

[0105] Thus, by performing relevant preprocessing on the original document text, the effectiveness of the words to be processed is improved, and the efficiency of subject term extraction is also increased.

[0106] In some implementations, S403 includes:

[0107] The topic relevance between each unprocessed word and the anchor word in the document is calculated based on the topic word extraction model.

[0108] The words to be processed are sorted in descending order of their topic relevance.

[0109] Based on the anchor point weight values, a predetermined number of words to be processed are extracted according to the sorting order of the words to be processed to obtain the topic words.

[0110] In some implementations, the processor is used to calculate the topic relevance between each word to be processed and the anchor word in the document according to the topic word extraction model, and to sort the words to be processed in descending order according to the numerical value of the topic relevance, and to extract a predetermined number of words to be processed according to the anchor weight value in the sorting order of the words to be processed to obtain the topic words.

[0111] Specifically, the correlation between each word to be processed and the anchor word can be calculated based on the topic word extraction model, and the probability value corresponding to the correlation can be used as the topic relevance of the word to be processed. Furthermore, the topic relevance can be used as a criterion for determining whether the word to be processed can be used as a topic word for extraction. In some examples, the words to be processed can be sorted in descending order according to the topic relevance, and then the words to be processed can be extracted as topic words.

[0112] In one example, keywords with a relevance greater than or equal to the anchor weight value can be filtered out, and then a predetermined number of keywords can be extracted from these keywords in descending order of relevance to obtain the topic keywords. Alternatively, a predetermined number of keywords can be extracted from the list of keywords in descending order of relevance to obtain the topic keywords. The predetermined number can be fixed, such as 15 or 20, or can be adjusted by the user before or after the topic keyword extraction.

[0113] In some examples, users can also modify the anchor weight value and the desired number of keywords to be extracted during the keyword extraction process. For instance, if the set anchor weight value is too high, fewer keywords may be extracted, even fewer than the predetermined number. In this case, the user can appropriately reduce the anchor weight value to extract more keywords. Of course, users can also reduce the number of keywords by adjusting the predetermined number.

[0114] In this way, keyword terms can be extracted based on the relevance between the target term and the anchor term.

[0115] The following example illustrates the process of generating the word selection list:

[0116] First, training text data is obtained as the model's dataset to train the model for generating word vectors. Then, the trained model is used to process the original text to obtain word vectors. For example, a dataset of Chinese academic papers in a specific field can be collected to pre-train a word2vector model for generating word vectors.

[0117] Furthermore, relevant word segmentation algorithms, such as the BiLSTM model, are used on the original text's title to divide it into a sequence of word units. For example... Figure 3 As shown, in the BiLSTM model, the BME annotation method is used for training sample labeling, where B, M, and E represent the beginning, middle, and end of a word, respectively. The Softmax layer outputs the probability of each label, and the label with the highest probability is selected as the predicted label.

[0118] Stop words and invalid words are filtered out based on the list of prohibited words, and words with the same form are initially deduplicated, retaining nouns and verbs in the word segmentation results.

[0119] Pointwise Mutual Information (PMI) values ​​are calculated for all words in the retained segmentation results. The specific formula is as follows: Where C(w) i ,w j The PMI value represents the frequency of co-occurrence of two words within a sliding window, and N represents the number of sliding windows. The sliding window represents the length of a sentence extracted from the original text each time, and its size can be treated as a manually adjustable parameter. A pre-trained word vector generation model is used to embed all words into vectors, and the PMI value is used as the weight of the edge between two words, thereby generating a word graph.

[0120] Based on the word graph, the retained words are deduplicated by synonyms. Specifically, all words are paired into word pairs. The word graph is used to calculate the weighted average of the two selected words and their first-order neighbors, and the cosine distance is calculated. If the distance is greater than a predetermined value, such as 0.5, the two words are considered to be synonyms. The word with the highest frequency is retained and used as the final candidate word, while synonyms are filtered out.

[0121] Please see Figure 4 This application also provides a server, including a processor 10 and a memory 20. The memory 20 is communicatively connected to the processor 10, and stores instructions. The processor 10 can implement the aforementioned text processing method by executing these instructions.

[0122] The server provided in this application can execute instructions stored in memory 20 through processor 10 to implement the above-mentioned text processing method. The effects that can be achieved by implementing this method have been discussed in detail above and will not be repeated here.

[0123] Please see Figure 5 This application provides a server that, through a human-computer interaction UI interface of a terminal or application communicating with the server, can retrieve documents with relevant content based on keywords input by the user and display text titles, such as... Figure 5 As shown, users can enter the keyword "object detection" and retrieve information. The server returns titles such as "Similar Image Matching Method Based on Deep Learning," "Interface Element Detection Method, Device, and Equipment," and "Low-Power Fast Image Target Detection Method Based on Deep Learning." Users can select the desired titles by checking the corresponding boxes. Furthermore, the server can perform operations such as word segmentation, deduplication, and filtering on the text content within the titles to obtain results like... Figure 6 The system generates a list of candidate words and selects one or more as anchor words based on user input. Then, it extracts the theme from the original text based on the anchor words to obtain the topic words.

[0124] Specifically, the server can use a topic word extraction model to extract topic words from the original text based on anchor words, thereby generating, for example, topic words such as anchor words. Figure 7 The keyword cloud shown, or the output as follows Figure 8 A list of keywords is provided for users to browse.

[0125] This application also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the above-described text processing method.

[0126] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0127] Furthermore, the descriptions of "words" or "terms" in this specification are for illustrative purposes only. No limitations are placed on the length or type of words or terms. In some specific embodiments, words may be replaced with similar statements or similar phrases.

[0128] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0129] Although this application has been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A text processing method, characterized in that, include: Perform word segmentation on the original text title; The segmented data is then filtered. Based on the filtered data, a list of candidate words is generated. In response to the user's input based on the selection of the candidate word list, anchor words are obtained, which are used to represent the user's topic word extraction preferences; Obtain a preset initial model, and configure the initial model according to the anchor weight value and the anchor word to obtain the topic word extraction model of the original text; Perform word segmentation on the document body of the original text; The data after the word segmentation operation is filtered out according to the preset list of prohibited words to obtain the words to be processed; The topic relevance between each word to be processed in the document and the anchor word is calculated based on the topic word extraction model. The words to be processed are sorted in descending order of their topic relevance scores. Based on the anchor point weight value, a predetermined number of words to be processed are extracted according to the sorting order of the words to be processed to obtain the topic words.

2. The processing method according to claim 1, characterized in that, The filtering process for the segmented data includes: The candidate words are obtained by filtering out stop words and invalid words from the segmented data according to the preset list of prohibited words, and the list of prohibited words includes the preset stop words and invalid words.

3. The processing method according to claim 2, characterized in that, The filtering process for the segmented data further includes: The data after filtering is subjected to the first deduplication process to remove identical word segments.

4. The processing method according to claim 3, characterized in that, The filtering process for the segmented data further includes: The data after the first removal process is subjected to a second deduplication process to filter out words with the same semantic meaning from the word segmentation, thereby obtaining the candidate words.

5. A server, characterized in that, include: A processor and a memory communicatively connected to the processor, wherein the memory stores instructions that the processor can implement the text processing method according to any one of claims 1 to 4 by executing the instructions.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the text processing method according to any one of claims 1 to 4.

Images