Text processing method and device, text classification method and device, equipment and storage medium

Abstract

The invention relates to a text processing method and device, a text classification method and device, equipment and a storage medium. The text classification method comprises the steps of obtaining a to-be-processed text; when the length of the to-be-processed text is greater than a preset length, extracting a first sub-text with a preset length from the to-be-processed text; under the condition that the second sub-text comprises the preset characters, splicing the preset characters and a plurality of characters in the first sub-text to obtain a first target spliced text with a preset length; wherein the second sub-text is the text, except the first sub-text, in the to-be-processed text. According to the method, the problem that the word number of the long text does not meet the language model requirement is solved, and the key characters representing the text core content and the first target spliced text of the subject name needing to be monitored can be intercepted from the long text to train the model, so that the performance of the model is improved; therefore, the language model obtained by training has higher accuracy when the text is classified.

Description

technical field [0001] The present disclosure relates to the technical field of the Internet, and in particular to a text processing method, a text classification method, a device, a device, and a storage medium. Background technique [0002] Due to the openness and communication characteristics of the Internet, it is necessary to obtain network public opinion analysis reports for network public opinion monitoring. Public opinion analysis platforms generally obtain various comments, articles, news, etc. from the Internet, and then classify the comments, articles and other texts . Since most of the texts on the network are long texts with a large number of words, and the current machine learning algorithm is limited by the machine memory and hardware configuration, it is impossible to train the entire content of long texts to obtain a classification model. Therefore, when long text is input to the language model for training and classification, it is often necessary to prepr...

AI Technical Summary

Problems solved by technology

[0004] The present disclosure provides a text processing method, a text classification method, a device, a device, and a storage medium, so as to at least solve the problem of high manual maintenance costs or high performance of the language model obtained by training when long text is used to train the language model in the related art. Ideal enough, the problem of low accuracy in classifying text

Images