Text topic analysis method, device, electronic device and storage medium

Abstract

The present invention provides a text topic analysis method, device, electronic device and storage medium, wherein the method includes: determining a number of analysis topics to be implemented and the corresponding analysis result label range; obtaining a training sample set containing multiple sample texts; each sample text may correspond to at least one analysis topic; determining the analysis result label of each sample text on the corresponding analysis topic; the analysis result label is located within the analysis result label range of the corresponding analysis topic; taking each sample text and the corresponding analysis topic as input, and the analysis result label as output, training the XLM‑RoBERTa network to obtain an XLM‑RoBERTa model; inputting the text to be evaluated and the target analysis topic into the XLM‑RoBERTa model, and obtaining the target analysis result label output by the XLM‑RoBERTa model. This solution can perform topic analysis on text according to different requirements, thereby improving the flexibility of topic analysis.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of natural language processing technology, and in particular to a text topic analysis method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of mobile Internet and information technology, the amount of data is also increasing rapidly. Massive data needs to be processed and analyzed, especially text data, which contains a huge amount of information. The demand for intelligent text topic analysis by governments, enterprises and individuals is growing. Therefore, natural language processing technology has been further developed.

[0003] In the prior art, the language models widely used in the field of natural language processing research include: Elmo, an advanced word vector based on recurrent neural network (RNN), GPT (Generative Pre-Training) model based on Transformer, and language representation model (Bidirectional Encoder Representation from Transformers, BERT). Although these language models, as text mining methods, can effectively extract text features and discover potential semantic themes in text data, they can only confirm the topic of the input text content and cannot be analyzed on demand, and have poor flexibility. Summary of the invention

[0004] Based on the problems of the prior art, the embodiments of the present invention provide a text topic analysis method, device, electronic device and storage medium, which can perform topic analysis on text according to different requirements and improve the flexibility of topic analysis.

[0005] In a first aspect, an embodiment of the present invention provides a text topic analysis method, comprising:

[0006] Determine a number of analysis topics to be implemented and a range of analysis result labels corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels;

[0007] Obtain a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic;

[0008] Determine an analysis result label of each sample text on a corresponding analysis topic; the analysis result label is within a range of analysis result labels of the corresponding analysis topic;

[0009] Taking each sample text and the corresponding analysis topic in the training sample set as input, taking the analysis result label of the input sample text on the input analysis topic as output, training the pre-built XLM-RoBERTa network, and obtaining a trained XLM-RoBERTa model;

[0010] The text to be evaluated and the target analysis topic are input into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

[0011] Preferably, the sample text in the training sample set is obtained from at least one of the following datasets: BookCorpus dataset, Wikipedia English dataset, CC-NEWS dataset, openwebtext dataset and stories dataset.

[0012] Preferably, the number of languages ​​used in the corpus in the sample text is at least one.

[0013] Preferably, the training of the pre-built XLM-RoBERTa network includes:

[0014] The XLM-RoBERTa network is used to perform text cleaning on the input sample text, and the sample text after text cleaning is segmented, and the segmented sample text is encoded into a tensor according to the segmentation result; and the encoded tensor, the input analysis topic and the corresponding analysis result label are used to perform feature learning on the sample text to adjust the parameters in the XLM-RoBERTa network.

[0015] Preferably, before encoding the word segmented sample text into a tensor according to the word segmentation result, the method further includes:

[0016] A part-of-speech tag is marked for each word, and words that do not contribute to text features are deleted according to the part-of-speech tag marked for each word, so as to perform the encoding into a tensor using the remaining words.

[0017] Preferably, the step of performing feature learning on the sample text using the encoded tensor, the input analysis topic and the corresponding analysis result label includes:

[0018] The selection ratio of each part-of-speech word is determined according to the input analysis topic, and feature selection is performed on each encoded tensor according to the selection ratio, so as to use the selected tensor as a learning feature for the sample text; wherein the ratio of the feature words of each part of speech corresponding to the selected tensor is the same as or close to the selection ratio.

[0019] Preferably, before performing feature learning on the sample text using the encoded tensor, the input analysis subject and the corresponding analysis result label, the method further includes:

[0020] The sample text is subjected to dynamic mask processing, and feature learning of the sample text is performed using a tensor encoded from the sample text after each mask processing; the feature learning does not include a mask prediction task.

[0021] In a second aspect, an embodiment of the present invention further provides a text topic analysis device, comprising:

[0022] A first determination unit is used to determine a number of analysis topics to be implemented and a range of analysis result labels corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels;

[0023] An acquisition unit, used to acquire a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic;

[0024] A second determining unit is used to determine an analysis result label of each sample text on a corresponding analysis topic; the analysis result label is within a range of analysis result labels of the corresponding analysis topic;

[0025] A training unit, used to take each sample text and the corresponding analysis topic in the training sample set as input, take the analysis result label of the input sample text on the input analysis topic as output, train the pre-constructed XLM-RoBERTa network, and obtain a trained XLM-RoBERTa model;

[0026] The analysis unit is used to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

[0027] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0028] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0029] The embodiment of the present invention provides a text topic analysis method, device, electronic device and storage medium, which adopts XLM-RoBERTa network for model training, can identify text streams composed of any number of sentences, and when training the XLM-RoBERTa model for text topic analysis, not only the sample text is used as input, but also the analysis topic required for the sample text is used as input, and the analysis result label of the sample text corresponding to the analysis topic is used as output, so that the constructed network can learn the characteristics of the sample text on the analysis topic, so that the analysis result is the same as the output analysis result label, so that when it is necessary to analyze different analysis topics of the text to be evaluated, it is only necessary to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model. It can be seen that this scheme can perform topic analysis on text according to different needs, thereby improving the flexibility of topic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flow chart of a text topic analysis method provided by one embodiment of the present invention;

[0032] Figure 2 is a flow chart of a model training method provided by one embodiment of the present invention;

[0033] Figure 3 is a hardware architecture diagram of an electronic device provided by an embodiment of the present invention;

[0034] Figure 4 It is a structural diagram of a text topic analysis device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] Among the widely used language models, the Transformer model can directly calculate the correlation between each word without passing through the hidden layer, and can perform parallel calculations to fully utilize GPU resources. However, its acquisition of local information is not as strong as that of RNN and convolutional neural networks. In addition, the final output layer of the Transformer model has no direct connection with the previous Transformer layer, and the gradient flow back is blocked by the layer normalization module, resulting in the disappearance of the top-level gradient.

[0037] Although the BERT model can obtain context-related bidirectional feature representations, it performs poorly in generation tasks and cannot perform document-level analysis tasks. It is only suitable for analyzing sentences and paragraphs.

[0038] In addition, the above models cannot analyze texts on different topics as needed, and have poor flexibility. Based on this, a text analysis model based on XLM-RoBERTa can be developed, which takes text and analysis topics as input and outputs the analysis result labels corresponding to the analysis topics for model training, so that the model can analyze text topics according to different needs.

[0039] The specific implementation of the above concept is described below.

[0040] Please refer to Figure 1 , an embodiment of the present invention provides a text topic analysis method, the method comprising:

[0041] Step 100, determining a number of analysis topics to be implemented and an analysis result label range corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels;

[0042] Step 102, obtaining a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic;

[0043] Step 104, determining the analysis result label of each sample text on the corresponding analysis topic; the analysis result label is within the analysis result label range of the corresponding analysis topic;

[0044] Step 106, taking each sample text and the corresponding analysis topic in the training sample set as input, taking the analysis result label of the input sample text on the input analysis topic as output, training the pre-built XLM-RoBERTa network, and obtaining a trained XLM-RoBERTa model;

[0045] Step 108, input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

[0046] In the embodiment of the present invention, the XLM-RoBERTa network is used for model training, which can recognize a text stream composed of any number of sentences, and when training the XLM-RoBERTa model for text topic analysis, not only the sample text is used as input, but also the analysis topic required for the sample text is used as input, and the analysis result label of the sample text corresponding to the analysis topic is used as output, so that the constructed network can learn the characteristics of the sample text on the analysis topic, so that the analysis result is the same as the output analysis result label, so that when it is necessary to analyze different analysis topics of the text to be evaluated, it is only necessary to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model. It can be seen that this solution can perform topic analysis on text according to different needs, thereby improving the flexibility of topic analysis.

[0047] Described below Figure 1 How the various steps are performed.

[0048] First, with respect to step 100, a number of analysis topics to be implemented and an analysis result label range corresponding to each analysis topic are determined; the analysis result label range includes a plurality of analysis result labels.

[0049] The text file contains a text stream composed of any number of sentences, and these text streams have corresponding topics after semantic translation. Furthermore, in one embodiment of the present invention, different topic analyses can be performed on the same text file, and analysis result labels of the text file on different analysis topics can be obtained. For example, for a text file, when analyzing the topic of "focus on topics", the analysis result label of the text file can be "education"; when analyzing the topic of "political tendency", the analysis result label of the text file can be "democratic tendency". It can be seen that different topic analyses can be performed on the same text file, and when performing different topic analyses, analysis result labels of corresponding analysis topics can be obtained.

[0050] Currently, by training the model, the model can classify text files by subject and output the subject to which the text files belong. In other words, it is not possible to perform subject analysis on text files on demand. The difference of the present invention is that it is possible to analyze text files with different analysis subjects according to different needs, and the output analysis result label also corresponds to the analysis subject, so as to improve the flexibility of subject analysis.

[0051] In order to train a model that can perform topic analysis on text files on demand, it is necessary to determine the number of analysis topics that need to be implemented and the range of analysis result labels corresponding to each analysis topic, so that the analysis result labels output by the model are within the corresponding range.

[0052] For example, the analysis topics to be implemented are: focus topics and political tendencies; when the analysis topic is focus topics, the corresponding analysis result label ranges include: education, medical care, politics, and people's livelihood; when the analysis topic is political tendencies, the corresponding analysis result label ranges include: democratic tendencies, civilian tendencies, centralized tendencies, and powerful tendencies.

[0053] Then, step 102 "obtaining a training sample set comprising a plurality of sample texts; each sample text may correspond to at least one analysis topic" and step 104 "determining an analysis result label for each sample text on the corresponding analysis topic; the analysis result label is within the analysis result label range of the corresponding analysis topic" are explained simultaneously.

[0054] In the embodiment of the present invention, when the number of analysis topics to be implemented is large, there are some sample texts that only have analysis value for some analysis topics, but not for other analysis topics. In this case, there is no need to use the part of the analysis topics that have no analysis value as the analysis topics of these sample texts to interfere with the feature learning of the model. Therefore, in order to improve the feature learning effect of the model and ensure that the parameter adjustment in the model is optimal, each sample text in the training sample set can correspond to at least one analysis topic among the several analysis topics, and the at least one analysis topic corresponding to the sample text is an analysis topic that the sample text has analysis value.

[0055] When determining the analysis topic corresponding to the sample text, it can be manually determined and then input to ensure the accuracy of determining the analysis topic corresponding to the sample text.

[0056] In one embodiment of the present invention, in order to improve the model training effect, more high-quality training data can be used to form a training sample set. Specifically, the sample text in the training sample set is obtained from at least one of the following data sets: BookCorpus data set, Wikipedia English data set, CC-NEWS data set, openwebtext data set and stories data set.

[0057] Among them, the BookCorpus dataset and the Wikipedia English dataset are the original data used for BERT training; the CC-NEWS dataset contains English news articles; the openwebtext dataset is an open source clone of the WebText corpus, which contains web page content crawled from shared links on the Reddit website; the stories dataset contains a subset of the CommonCrawl data, which has been filtered to match the storytelling style of Winograd schemas.

[0058] The training sample set can be obtained from the above data set by downloading public data or crawling the corpus to form sample texts, and then obtaining the training sample set.

[0059] When the training sample set obtains sample text from the above five data sets, the data volume of the training sample set in this embodiment is more than ten times the data volume of BERT, thereby improving the quality of model training.

[0060] In one embodiment of the present invention, in order to achieve cross-language analysis, the number of languages ​​in the training sample set can be increased, and the cross-language representation can be trained in a self-supervised manner to solve the multilingual curse problem. Specifically, the number of languages ​​used in the corpus in the sample text is at least one.

[0061] When the current XLM-RoBERTa model is trained, each language is taken as a batch and trained in batches, that is, sample texts of the same batch corresponding to the same language are input into the XLM-RoBERTa network for training. After the training is completed, another batch of sample texts of the same language (different from the language of the previous batch) is used to continue training to obtain an XLM-RoBERTa model for multilingual text analysis. In the embodiment of the present invention, when the XLM-RoBERTa model is trained, the corpus in the same sample text can use one language or multiple languages. When multiple languages ​​are mixed in the same sample text, the text stream of the corresponding language of the sample text can be translated based on the corresponding language, and the translation result can be semantically analyzed, and then the corresponding topic analysis can be performed.

[0062] In an embodiment of the present invention, after obtaining the training sample set, it is also necessary to determine the analysis result label of each sample text on the corresponding analysis topic, and in order to ensure that the XLM-RoBERTa model can output the analysis result label within the specified range, the analysis result label of the determined sample text on the corresponding analysis topic must be within the analysis result label range of the corresponding analysis topic.

[0063] When a sample text in the training sample set corresponds to an analysis topic, an analysis result label on the analysis topic can be determined for the sample text;

[0064] When the sample text in the training sample set corresponds to multiple analysis topics, it is necessary to determine the analysis result label on each analysis topic for the sample text.

[0065] For example, sample text 1 corresponds to analysis topic 1 and analysis topic 2, then the analysis result label 11 corresponding to analysis topic 1 is determined for sample text 1, and the analysis result label 21 corresponding to analysis topic 2 is determined for sample text 1. Since sample text 1 corresponds to two analysis topics, each analysis topic can be used for training the XLM-RoBERTa model, so this can be regarded as an expansion of the training sample set.

[0066] Since when a sample text corresponds to multiple analysis topics, by determining the analysis result label on each analysis topic for the sample text, it is equivalent to expanding the sample text into multiple sample texts corresponding to the multiple analysis topics one by one. The training sample set (referred to as A) obtained by this expansion method is compared with the method in which one sample text corresponds to one analysis topic in the training sample set (referred to as B). Even if the number of sample texts in A and B is the same after the final expansion in this embodiment, the XLM-RoBERTa model trained using the sample text of A will have a higher accuracy rate when analyzing different analysis topics for the same text compared to the XLM-RoBERTa model trained using the sample text of B. This is because there are the same sample texts corresponding to different analysis topics in A, and the XLM-RoBERTa model will learn this feature during the training process, thereby improving the accuracy of analyzing different analysis topics for the same text.

[0067] Next, step 106 is explained. Each sample text and the corresponding analysis topic in the training sample set are taken as input respectively, and the analysis result label of the input sample text on the input analysis topic is taken as output. The pre-constructed XLM-RoBERTa network is trained to obtain a trained XLM-RoBERTa model.

[0068] In one embodiment of the present invention, when training the XLM-RoBERTa network, sample text and corresponding analysis topics can be directly input. After the XLM-RoBERTa network processes the input sample text, feature learning is performed to improve the accuracy of the training results and improve the feature learning effect.

[0069] Specifically, in one embodiment of the present invention, please refer to Figure 2 , which trains the pre-built XLM-RoBERTa network, can include:

[0070] S1. Use the XLM-RoBERTa network to clean the input sample text;

[0071] S2. Tokenize the sample text after text cleaning;

[0072] S3. Encode the tokenized sample text into a tensor according to the tokenization result;

[0073] S4. Use the tensor obtained by encoding, the input analysis topic, and the corresponding analysis result label to perform feature learning on the sample text, so as to adjust the parameters in the XLM-RoBERTa network.

[0074] In step S1, there is noise data in the sample text. By cleaning the sample text, useful data can be retained and noise data can be deleted. The text cleaning method in this step may include at least one of deduplication, alignment, deletion, and annotation.

[0075] In steps S2 and S3, when tokenizing the sample text, a rule-based tokenization method or a statistics-based tokenization method can be used for tokenization. And the way of encoding into a tensor can adopt one-hot and embedding encoding methods.

[0076] In an embodiment of the present invention, before step S3, it may further include: annotating a part-of-speech tag for each word, and deleting the words that do not contribute to the text features according to the annotated part-of-speech tags of each word, so as to use the remaining words to perform the encoding into a tensor.

[0077] Among them, the part of speech may include nouns, verbs, adjectives, etc. When annotating a part-of-speech tag for each word, a maximum entropy part-of-speech tagging method, a hidden Markov model (HMM) part-of-speech tagging method, etc. can be used. And deleting the words that do not contribute to the text features is to improve the contribution of the words based on which the feature learning is performed in the subsequent process to the text features, thereby improving the feature learning effect and the accuracy of parameter adjustment.

[0078] In one implementation, the contribution degree of a word to the text features can be determined based on the part-of-speech tag of the word. For example, punctuation marks, modal particles, "de" (的 in Chinese), etc., these words contribute less to the text features. Therefore, these words can be deleted.

[0079] In an embodiment of the present invention, in order to improve the accuracy of model training and enable the model to learn different language representations, before step S4, it may further include: performing dynamic masking processing on the sample text, and using the tensor encoded from the sample text after each masking processing to perform feature learning on the sample text; the feature learning does not include the prediction task of the mask.

[0080] Specifically, the dynamic mask processing method can be to randomly extract a set proportion of bytes from the text stream in the sample text, and then mask the set proportion of bytes. Each mask processing is a random extraction process, so a new sample text can be formed after each mask processing, thereby further realizing the expansion of the training sample set, and using the masked sample text to train the XLM-RoBERTa network, so that the XLM-RoBERTa network can continuously learn features and gradually adapt to different masking strategies, so as to learn different language representations.

[0081] Preferably, the set ratio is no greater than 15%.

[0082] In addition, since the trained XLM-RoBERTa model is used to analyze the corresponding analysis topics of the text, when training after masking, there is no need to predict the masks in the masked sample text. Instead, after the sample text is masked, it is still possible to learn the characteristics of the sample text on the corresponding analysis topic and then output the analysis result label of the corresponding analysis topic.

[0083] In combination with step 102, the corpus in a sample text can use multiple languages, and the present embodiment can still train XLM-RoBERTa, because there is no need to predict the mask after masking, and it is only necessary to use the remaining text stream after masking to analyze the analysis result label of the corresponding analysis topic. Therefore, when the corpus in the sample text uses multiple languages, the sample text obtained by dynamic masking of the sample text can be used to train an accurate XLM-RoBERTa model that can perform different analysis topics as needed.

[0084] It should be noted that the mask processing process can be performed before S3 or after S3. It only needs to ensure that when S4 is executed, feature learning is performed based on the tensor corresponding to the sample text after mask processing.

[0085] In step S4, in one embodiment, the feature learning of the sample text using the encoded tensor, the input analysis topic and the corresponding analysis result label can include: determining the selection ratio of each part-of-speech word according to the input analysis topic, and performing feature selection on each encoded tensor according to the selection ratio, so as to use the selected tensor as the learning feature of the sample text; wherein the ratio of the feature words of each part of speech corresponding to the selected tensor is the same as or similar to the selection ratio.

[0086] Since the selected tensor is used as the learning feature of the sample text after feature selection, the selected tensor needs to be adapted to the analysis topic. The higher the adaptability, the higher the accuracy when performing the corresponding topic analysis.

[0087] Specifically, a correspondence between the analysis topic and the selection ratio of each part of speech word may be established in advance, and the process of establishing the correspondence may include:

[0088] Get several sample texts;

[0089] For each sample text obtained, the following steps are performed: extracting the central idea of ​​the sample text on the corresponding analysis topic to obtain a plurality of extracted words, and classifying the plurality of words into parts of speech;

[0090] The number of words of each part of speech corresponding to the same analysis topic in several sample texts is counted to obtain the ratio relationship of the words of each part of speech corresponding to each analysis topic, and the ratio relationship is determined as the selection ratio corresponding to the analysis topic.

[0091] For example, for analysis topic 1, the number of words in the lyrics corresponding to the analysis topic 1 in several sample texts is statistically obtained as follows: 1000 nouns, 800 verbs, and 300 adjectives. The selection ratio of each part of speech word for the analysis topic 1 is: noun: verb: adjective = 10:8:3. In other words, when performing feature selection, the corresponding part of speech words can be selected according to the same selection ratio or a similar selection ratio, so that the target feature subset obtained after feature selection is more suitable for the analysis topic, and the accuracy of the analysis result is improved.

[0092] The similar selection ratio may be a ratio that is within a set deviation range from the selection ratio.

[0093] Step 206, taking the target feature subset and the corresponding analysis topic of the sample text as input respectively to train the pre-constructed XLM-RoBERTa network.

[0094] In an embodiment of the present invention, the constructed XLM-RoBERTa network may include three fully connected layers to improve the accuracy of the analysis results.

[0095] During the training process, larger model parameters can be used. For example, 1024 V100 GPUs can be used for training for one day. By increasing the number of training steps, that is, expanding the mini_batch parameter, the model optimization rate and model performance can be improved.

[0096] Finally, step 108 is explained, where the text to be evaluated and the target analysis topic are input into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

[0097] After the XLM-RoBERTa model is trained, the text to be evaluated can be analyzed on different analysis topics as needed. For example, a person's social post is identified as the text to be evaluated. By analyzing the text to be evaluated on different analysis topics, the person's focus topics, political inclinations, and other results can be obtained.

[0098] During the training process, the sample text can also be used as output, and the analysis result label can be returned at the end of the sample text. The output sample text can be the input sample text or the sample text after text cleaning, so as to facilitate the next statistical analysis. For example, the part-of-speech ratio of words in the output sample text can be used to update the selection ratio of words of each part of speech.

[0099] like Figure 3 , Figure 4 As shown, an embodiment of the present invention provides a text topic analysis device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 3 As shown, it is a hardware architecture diagram of an electronic device where a text topic analysis device provided by an embodiment of the present invention is located, except Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4 As shown, as a device in a logical sense, the CPU of the electronic device in which it is located reads the corresponding computer program in the non-volatile memory into the memory and runs it. A text topic analysis device provided in this embodiment includes:

[0100] The first determination unit 401 is used to determine a number of analysis topics to be implemented and an analysis result label range corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels;

[0101] The acquisition unit 402 is used to acquire a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic;

[0102] The second determining unit 403 is used to determine the analysis result label of each sample text on the corresponding analysis topic; the analysis result label is within the analysis result label range of the corresponding analysis topic;

[0103] The training unit 404 is used to take each sample text and the corresponding analysis topic in the training sample set as input, take the analysis result label of the input sample text on the input analysis topic as output, train the pre-constructed XLM-RoBERTa network, and obtain a trained XLM-RoBERTa model;

[0104] The analysis unit 405 is used to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

[0105] In one embodiment of the present invention, the sample text in the training sample set is obtained from at least one of the following datasets: BookCorpus dataset, Wikipedia English dataset, CC-NEWS dataset, openwebtext dataset and stories dataset.

[0106] In one embodiment of the present invention, the number of languages ​​used in the corpus in the sample text is at least one.

[0107] In one embodiment of the present invention, the training unit 404 is specifically used to: use the XLM-RoBERTa network to perform text cleaning on the input sample text, segment the sample text after text cleaning, and encode the segmented sample text into a tensor according to the segmentation result; and use the encoded tensor, the input analysis topic and the corresponding analysis result label to perform feature learning on the sample text to adjust the parameters in the XLM-RoBERTa network.

[0108] In one embodiment of the present invention, the training unit 404 is further used to label each word with a part-of-speech tag before encoding the word-segmented sample text into a tensor according to the word segmentation result, and delete words that do not contribute to the text features according to the part-of-speech tag labeled for each word, so as to use the remaining words to perform the encoding into a tensor.

[0109] In one embodiment of the present invention, the training unit 404, when executing the feature learning of the sample text using the encoded tensor, the input analysis topic and the corresponding analysis result label, specifically includes: determining the selection ratio of each part-of-speech word according to the input analysis topic, and performing feature selection on each encoded tensor according to the selection ratio, so as to use the selected tensor as the learning feature of the sample text; wherein the ratio of the feature words of each part of speech corresponding to the selected tensor is the same as or similar to the selection ratio.

[0110] In one embodiment of the present invention, the training unit 404 is further used to perform dynamic mask processing on the sample text before performing feature learning on the sample text using the encoded tensor, the input analysis topic and the corresponding analysis result label, and perform feature learning on the sample text using the tensor encoded from the sample text after each mask processing; the feature learning does not include the task of predicting the mask.

[0111] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a text topic analysis device. In other embodiments of the present invention, a text topic analysis device may include more or fewer components than those shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0112] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0113] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a text topic analysis method in any embodiment of the present invention is implemented.

[0114] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a text topic analysis method in any embodiment of the present invention.

[0115] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0116] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0117] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0118] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0119] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0120] Each embodiment of the present invention has at least the following beneficial effects:

[0121] 1. In one embodiment of the present invention, an XLM-RoBERTa network is used for model training, which can identify a text stream composed of any number of sentences, and when training an XLM-RoBERTa model for text topic analysis, not only the sample text is used as input, but also the analysis topic required for the sample text is used as input, and the analysis result label of the sample text corresponding to the analysis topic is used as output, so that the constructed network can learn the characteristics of the sample text on the analysis topic, so that the analysis result is the same as the output analysis result label, so that when it is necessary to analyze different analysis topics of the text to be evaluated, it is only necessary to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model. It can be seen that this solution can perform topic analysis on text according to different needs, thereby improving the flexibility of topic analysis.

[0122] 2. In one embodiment of the present invention, when the number of analysis topics to be implemented is large, there are some sample texts that only have analysis value for some analysis topics, but not for other analysis topics. In this case, there is no need to use the part of the analysis topic that has no analysis value as the analysis topic of these sample texts to interfere with the feature learning of the model. Therefore, in order to improve the feature learning effect of the model and ensure that the parameter adjustment in the model is optimal, each sample text in the training sample set can correspond to at least one analysis topic among the several analysis topics, and the at least one analysis topic corresponding to the sample text is an analysis topic that the sample text has analysis value.

[0123] 3. In one embodiment of the present invention, when a sample text corresponds to multiple analysis topics, by determining the analysis result label on each analysis topic for the sample text, it is equivalent to expanding the sample text into multiple sample texts corresponding to the multiple analysis topics one by one. The training sample set (referred to as A) obtained by this expansion method is compared with the method in which one sample text corresponds to one analysis topic in the training sample set (referred to as B). Even if the number of sample texts in A and B is the same after the final expansion in this embodiment, the XLM-RoBERTa model trained using the sample text of A will have a higher accuracy rate when analyzing different analysis topics for the same text compared to the XLM-RoBERTa model trained using the sample text of B. This is because there are the same sample texts corresponding to different analysis topics in A, and the XLM-RoBERTa model will learn this feature during the training process, thereby improving the accuracy rate of analyzing different analysis topics for the same text.

[0124] 4. In one embodiment of the present invention, the corpus in a sample text may use multiple languages, and this embodiment can still train XLM-RoBERTa. This is because there is no need to predict the mask after masking. It is only necessary to use the remaining text stream after masking to analyze the analysis result label of the corresponding analysis topic. Therefore, when the corpus in the sample text uses multiple languages, the sample text obtained by dynamic masking of the sample text can be used to train an accurate XLM-RoBERTa model that can perform different analysis topics as needed.

[0125] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0126] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text topic analysis method, characterized in that: include: Determine a number of analysis topics to be implemented and a range of analysis result labels corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels; Obtain a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic; Determine an analysis result label of each sample text on a corresponding analysis topic; the analysis result label is within a range of analysis result labels of the corresponding analysis topic; Taking each sample text and the corresponding analysis topic in the training sample set as input, taking the analysis result label of the input sample text on the input analysis topic as output, training the pre-built XLM-RoBERTa network, and obtaining a trained XLM-RoBERTa model; The text to be evaluated and the target analysis topic are input into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

2. The method according to claim 1, characterized in that: The sample texts in the training sample set are obtained from at least one of the following datasets: BookCorpus dataset, Wikipedia English dataset, CC-NEWS dataset, openwebtext dataset and stories dataset.

3. The method according to claim 1 or 2, characterized in that: The number of languages ​​used in the corpus of the sample text is at least one.

4. The method according to claim 1, characterized in that: The training of the pre-built XLM-RoBERTa network includes: The XLM-RoBERTa network is used to perform text cleaning on the input sample text, and the sample text after text cleaning is segmented, and the segmented sample text is encoded into a tensor according to the segmentation result; and the encoded tensor, the input analysis topic and the corresponding analysis result label are used to perform feature learning on the sample text to adjust the parameters in the XLM-RoBERTa network.

5. The method according to claim 4, characterized in that Before encoding the word segmented sample text into a tensor according to the word segmentation result, the method further includes: A part-of-speech tag is marked for each word, and words that do not contribute to text features are deleted according to the part-of-speech tag marked for each word, so as to perform the encoding into a tensor using the remaining words.

6. The method according to claim 4 or 5, characterized in that: The method of performing feature learning on the sample text using the encoded tensor, the input analysis topic, and the corresponding analysis result label includes: The selection ratio of each part-of-speech word is determined according to the input analysis topic, and feature selection is performed on each encoded tensor according to the selection ratio, so as to use the selected tensor as a learning feature for the sample text; wherein the ratio of the feature words of each part of speech corresponding to the selected tensor is the same as or close to the selection ratio.

7. The method according to claim 4 or 5, characterized in that: Before performing feature learning on the sample text using the encoded tensor, the input analysis subject and the corresponding analysis result label, the method further includes: The sample text is subjected to dynamic mask processing, and feature learning of the sample text is performed using a tensor encoded from the sample text after each mask processing; the feature learning does not include a mask prediction task.

8. A text topic analysis device, characterized in that: include: A first determination unit is used to determine a number of analysis topics to be implemented and a range of analysis result labels corresponding to each analysis topic; the analysis result label range includes a plurality of analysis result labels; An acquisition unit, used to acquire a training sample set including a plurality of sample texts; each sample text may correspond to at least one analysis topic; A second determining unit is used to determine an analysis result label of each sample text on a corresponding analysis topic; the analysis result label is within a range of analysis result labels of the corresponding analysis topic; A training unit, used to take each sample text and the corresponding analysis topic in the training sample set as input, take the analysis result label of the input sample text on the input analysis topic as output, train the pre-constructed XLM-RoBERTa network, and obtain a trained XLM-RoBERTa model; The analysis unit is used to input the text to be evaluated and the target analysis topic into the XLM-RoBERTa model to obtain the target analysis result label output by the XLM-RoBERTa model.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images