Text classification method and device, computer device and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a text classification method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a text to be processed; performing cutting processing on a character sequence in the text to be processed according to a character type to obtain a cut word block; determining the position of the cut word block in the text to be processed, and obtaining a category identifier of the cut word block according to the category to which the cut word block belongs; obtaining feature data of the text to be processed according to the cut word block, the category identifier of the cut word block and the position of the cut word block in the text to be processed; and performing classification analysis on the text to be processed according to the feature data to obtain a classification result. Through fine cutting and sub-word cutting processing, and category identification of the cut word block, feature data with richer semantic knowledge is obtained, and the classification analysis on the text to be processed can obtain a more accurate classification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a text classification method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of information technology, the speed of information dissemination is getting faster and faster, and the dissemination channels are also more and more. In the platform where comments can be published, such as various social platforms, work publishing platforms, etc., there will be a need to identify whether the text of various advertisements is the target category.

[0003] In the traditional technology, the language processing model is generally used to identify the target category text in the comments based on the preset word table. However, the publisher of the target category text often uses homophonic characters, special characters and other ways to write in order to evade the content review of the platform. The variants of the target category text are very fast, and the forms are novel, and many special characters do not appear in the word table, so that the category of the text cannot be determined, and the recognition accuracy of the target category text is not high. SUMMARY

[0004] Therefore, it is necessary to provide a text classification method, device, computer equipment and storage medium capable of improving the recognition accuracy of the target category text in view of the above technical problems.

[0005] A text classification method, the method comprising:

[0006] obtaining a to-be-processed text;

[0007] performing segmentation processing on a character sequence in the to-be-processed text according to a character type to obtain a segmented word block;

[0008] determining a position of the segmented word block in the to-be-processed text, and obtaining a category identifier of the segmented word block according to a category to which the segmented word block belongs;

[0009] obtaining feature data of the to-be-processed text according to the segmented word block, the category identifier of the segmented word block and the position of the segmented word block in the to-be-processed text;

[0010] performing classification analysis on the to-be-processed text according to the feature data to obtain a classification result.

[0011] A text classification method, the method comprising:

[0012] The construction process of the BERT model comprises:

[0013] Training data carrying category labels is obtained to train the initial BERT model and obtain the teacher model. The feature data acquisition process of the initial BERT model includes word chunk segmentation, word chunk classification labeling, and word chunk position labeling.

[0014] The training data is predicted based on the teacher model to obtain the classification probability of the training data;

[0015] The initial BERT model is subjected to hierarchical compression to obtain a compressed BERT model;

[0016] Based on a preset loss function, the compressed BERT model is trained using training data carrying the classification probability and the category label to construct the BERT model.

[0017] A text classification device, the device comprising:

[0018] The text acquisition module is used to acquire the text to be processed.

[0019] The word segmentation module is used to segment the character sequence in the text to be processed according to the character type, and obtain segmented word blocks;

[0020] The category and location determination module is used to determine the location of the segmented word blocks in the text to be processed, and obtain the category identifier of the segmented word blocks according to the category to which the segmented word blocks belong;

[0021] The feature data processing module is used to obtain the feature data of the text to be processed based on the segmented word blocks, the category identifier of the segmented word blocks, and the position of the segmented word blocks in the text to be processed;

[0022] The classification analysis module is used to classify and analyze the text to be processed based on feature data, and obtain the classification results.

[0023] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0024] Get the text to be processed;

[0025] The character sequence in the text to be processed is segmented according to the character type to obtain segmented word blocks;

[0026] Determine the position of the segmented word block in the text to be processed, and obtain the category identifier of the segmented word block according to its category;

[0027] Based on the segmented word blocks, the category identifier of the segmented word blocks, and the position of the segmented word blocks in the text to be processed, the feature data of the text to be processed is obtained;

[0028] According to the feature data, the to-be-processed text is classified and analyzed to obtain a classification result.

[0029] A computer-readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0030] Obtaining a to-be-processed text;

[0031] Splitting a character sequence in the to-be-processed text according to a character type to obtain a split word block;

[0032] Determining a position of the split word block in the to-be-processed text, and obtaining a category identifier of the split word block according to a category to which the split word block belongs;

[0033] Obtaining feature data of the to-be-processed text according to the split word block, the category identifier of the split word block, and the position of the split word block in the to-be-processed text;

[0034] According to the feature data, the to-be-processed text is classified and analyzed to obtain a classification result.

[0035] The text classification method, device, computer device, and storage medium described above obtain a to-be-processed text, split a character sequence in the to-be-processed text according to a character type to obtain a split word block, implement fine splitting of characters in the to-be-processed text, determine a position of the split word block in the to-be-processed text, obtain a category identifier of the split word block according to a category to which the split word block belongs, obtain feature data of the to-be-processed text according to the split word block, the category identifier of the split word block, and the position of the split word block in the to-be-processed text, use category information of the characters to obtain feature data with richer semantic knowledge, and classify and analyze the to-be-processed text to obtain a more accurate classification result. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 An application environment diagram of a text classification method according to an embodiment;

[0037] Figure 2 A flowchart of a text classification method according to an embodiment;

[0038] Figure 3 A flowchart of a text classification method according to another embodiment;

[0039] Figure 4 A flowchart of a text classification method according to still another embodiment;

[0040] Figure 5 A flowchart of model training in a text classification method according to an embodiment;

[0041] Figure 6A flowchart of a text classification method of another embodiment;

[0042] Figure 7 A flowchart of a text classification method of another embodiment;

[0043] Figure 8 A text upload interface diagram of a text classification method of an embodiment;

[0044] Figure 9 A flowchart of a text classification method of an embodiment;

[0045] Figure 10 A model structure diagram of a text classification method of another embodiment;

[0046] Figure 11 A structural block diagram of a text classification device of an embodiment;

[0047] Figure 12 An internal structure diagram of a computer device of an embodiment. DETAILED DESCRIPTION

[0048] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0049] With the research and progress of artificial intelligence (AI) technology, artificial intelligence technology is researched and applied in many fields. Artificial intelligence is the theory, method, technology and application system for perceiving environment, obtaining knowledge and using knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0050] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware and software technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0051] The scheme provided by the embodiments of the present application relates to natural language processing and other technologies of artificial intelligence. Natural language processing is an important direction in the field of computer science and the field of artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e., the language used in daily life, so it is closely related to the research of linguistics.

[0052] The text classification method provided by the present application can be applied to the application environment as shown in Figure 1 . In the application environment, a terminal 102 is deployed with a text classification model. The terminal obtains a text to be processed. The character sequence in the text to be processed is segmented according to character types to obtain segmented word blocks. The position of the segmented word blocks in the text to be processed is determined, and the category identifiers of the segmented word blocks are obtained according to the categories to which the segmented word blocks belong. The feature data of the text to be processed is obtained according to the segmented word blocks, the category identifiers of the segmented word blocks, and the positions of the segmented word blocks in the text to be processed. The text to be processed is classified and analyzed according to the feature data to obtain a classification result, which is pushed to the display interface of the terminal. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones or tablet computers.

[0053] In another embodiment, the text classification model can also be deployed in a server, which is in communication connection with the terminal through a network. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0054] In an embodiment, as shown in Figure 2 , a text classification method is provided. The method is described by taking a terminal in Figure 1 as an example, which includes the following steps 202 to 210.

[0055] Step 202, obtaining a text to be processed.

[0056] The text to be processed refers to a text that needs to be classified by a model. In the embodiments, the classification of the text to be processed can be realized by computer data analysis, or can be realized based on a text classification model configured in the computer. The expected category of the text to be processed corresponds to one of the classification categories of the text classification model. For example, when the text classification model is a model for distinguishing whether it is an advertisement, the text to be processed is a text that needs to be identified whether it is an advertisement.

[0057] Step 204, segmenting the character sequence in the text to be processed according to character types to obtain segmented word blocks.

[0058] The character types include mathematical characters, English characters, Chinese characters, and other special characters (such as Japanese, Korean, and other language characters, and emoji, for example). A character sequence is a sequence of multiple characters. In an embodiment, a relatively short text to be processed can be regarded as a character sequence, and a relatively long text to be processed can be regarded as multiple character sequences. For example, the text to be processed can be roughly divided into multiple character sequences by a pre-set sequence division rule. The pre-set sequence division rule can be to divide character sequences according to punctuation marks (such as full stop, exclamation mark, and other punctuation marks representing the end of semantics), for example, a sentence ending with a full stop is divided into a character sequence. The pre-set sequence division rule can also be to divide character sequences based on the same character type. For example, multiple consecutive English words are divided into a sequence, and multiple consecutive Chinese characters are divided into a sequence. In other embodiments, the pre-set sequence division rule can also be to divide the text to be processed into sentences first, and then divide the text to be processed into character sequences, which is not limited herein.

[0059] The terminal has different processing manners for different character types in a character sequence. For example, Chinese sequences are segmented into words, and non-Chinese sequences are segmented into subwords. It should be noted that "word segmentation" here emphasizes the action of segmenting text, and the result of segmentation can be a single character or a word composed of multiple characters, and is not limited to only a "word" composed of multiple characters. Specifically, the granularity of word segmentation can be determined by parameter configuration according to a specific scenario. The granularity of word segmentation includes character granularity and word granularity.

[0060] In an embodiment, the terminal obtains a text to be segmented into words by identifying Chinese sequences in the text to be processed, and performs word segmentation on the text to be segmented into words according to the configured granularity parameter of word segmentation.

[0061] Subword segmentation (Subword in natural language processing, NLP) is for non-Chinese multi-character combined words, for example, English words can be divided into roots and affixes. The process of dividing a word into roots and affixes here is subword segmentation. For example, unigram can be divided into smaller units uni+gram, going can be divided into smaller units go+ing, and Learned can be divided into smaller units Learn+ed.

[0062] A segmented word block, also known as a WordPiece, is a character unit obtained by word segmentation and subword segmentation. In an embodiment, by performing character segmentation on Chinese sequences and subword segmentation on non-Chinese sequences, segmented word blocks such as mathematical characters, English characters, and Chinese characters can be obtained.

[0063] Step 206, determining the position of the segmented word block in the text to be processed, and obtaining the category identifier of the segmented word block according to the category to which the segmented word block belongs.

[0064] The position of the segmented word block in the text to be processed is in terms of the arrangement order of each segmented word block in the text to be processed. For example, assuming that a sentence is segmented into 10 segmented word blocks, the segmented word blocks have corresponding orders 1-10 respectively, and the order identifier of each segmented word block is both different and related, and can be used to identify the relationship between each segmented word block and other segmented word blocks.

[0065] The category to which the segmented word block belongs refers to the category corresponding to the segmentation result defined in advance. For example, Chinese characters, English characters, mathematical characters, etc., wherein the Chinese characters include commonly used Chinese characters in a preset word table, and uncommon Chinese characters not in the preset word table, such as traditional Chinese characters and rare Chinese characters. The roots, affixes, and common abbreviations of English words in English characters are common English characters, and a single letter among 26 common letters and English punctuation marks are uncommon English characters. The common combination results of multiple numbers in 0-9 in mathematical characters are common mathematical characters, and single numbers or mathematical symbols are uncommon mathematical characters.

[0066] According to the pre-defined classification category and the category to which the segmented word block belongs, the category identifier of the segmented word block is obtained. For example, common mathematical characters can be represented by “num” (short for number), and uncommon mathematical characters can be represented by “unk_num”. For another example, common Chinese characters can be represented by “zh”, and uncommon Chinese characters can be represented by “unk_zh”. Common English characters are represented by “letters”, and uncommon English characters are represented by “unk_letters”.

[0067] Step 208, obtaining the feature data of the text to be processed according to the segmented word block, the category identifier of the segmented word block, and the position of the segmented word block in the text to be processed.

[0068] The feature data refers to a feature representation of multiple dimensions for representing a certain data, which can be embodied in a feature vector or the like. In an embodiment, for each segmented word block, at least three dimensions of features are included, specifically including the meaning of the segmented word block itself, the meaning of a category represented by a category identifier of the segmented word block, and the contextual meaning represented by the position of the segmented word block in the text to be processed. By combining the feature meanings of multiple dimensions, richer semantic knowledge can be learned, which helps to improve the accuracy of the classification result in the classification analysis. In an embodiment, the feature data of the text to be processed can also be obtained by segmenting the text to be processed, determining the segment to which the segmented word block belongs, and then obtaining the segmented word block, the category identifier of the segmented word block, the segment to which the segmented word block belongs, and the position of the segmented word block in the text to be processed.

[0069] In an embodiment, the segmented word block can also be preprocessed, such as escape processing, for example, mapping a special character "Ô" in the text to "o" before inputting the model. Specifically, the escape processing can be determined according to the category identifiers of adjacent characters, for example, when the adjacent character of the special character "Ô" is a capital English letter, "Ô" can be mapped to "O", if the adjacent character is a lowercase English letter, "Ô" can be mapped to "o", and if the adjacent character is a number, "Ô" can be mapped to "0". By performing escape processing on the segmented word block, the meaning represented by the segmented word block in the text to be processed can be better represented, and the classification accuracy can be improved.

[0070] In step 210, the text to be processed is classified and analyzed according to the feature data, and a classification result is obtained.

[0071] The feature data is processed by feature fusion, which combines the processing results of each segmented word block and multi-dimensional semantics in the text to be processed. Based on the feature data of the text to be processed, the text to be processed is classified and analyzed, and thus a more accurate classification result is obtained.

[0072] The above text classification method, by obtaining the text to be processed, performing segmentation processing on the character sequence in the text to be processed according to the character type, obtaining the segmented word block, realizes the fine segmentation of the characters in the text to be processed, determines the position of the segmented word block in the text to be processed, and obtains the category identifier of the segmented word block according to the category to which the segmented word block belongs, obtains the feature data of the text to be processed according to the segmented word block, the category identifier of the segmented word block, and the position of the segmented word block in the text to be processed, uses the category information of the characters, can obtain feature data with richer semantic knowledge, and can obtain more accurate classification results by classifying and analyzing the text to be processed.

[0073] In an embodiment, as Figure 3As shown, the character sequence in the to-be-processed text is segmented according to the character type to obtain segmented word blocks, i.e., step 204, including steps 302 to 306.

[0074] In step 302, Chinese sequences and non-Chinese sequences in the to-be-processed text are identified.

[0075] In step 304, the Chinese sequences are segmented according to a preset Chinese segmentation granularity parameter to obtain Chinese word blocks.

[0076] A Chinese sequence refers to a sequence composed of multiple continuous Chinese characters (i.e., Hanzi), and a non-Chinese sequence refers to a sequence composed of multiple characters containing non-Chinese characters, which can be mathematical characters, English characters, or other language characters (such as Korean, Japanese, Latin, etc.). Based on the character attributes carried by each character in each to-be-processed text, it can be determined whether the character is Chinese, thereby identifying the Chinese sequences and non-Chinese sequences in the to-be-processed text.

[0077] Granularity refers to the thickness of data in the same dimension. The Chinese segmentation granularity parameter refers to a parameter for defining the unit size of word segmentation. The Chinese segmentation granularity parameter includes word granularity and character granularity. Through different Chinese segmentation granularity parameters, different word block segmentation of the Chinese sequences in the to-be-processed text can be achieved. The word granularity refers to that a Chinese sequence contains one Hanzi, and the character granularity refers to that a Chinese sequence can contain one or more than two Hanzi. Taking "go to school" as an example, through the segmentation method of word granularity, it can be segmented into "go", "learn", and "school", and through the segmentation method of character granularity, since "school" is a fixed collocation, the segmentation result is "go" and "school".

[0078] In one embodiment, the Chinese sequences in the to-be-processed text are segmented according to a preset Chinese segmentation granularity parameter to obtain Chinese word blocks, including two ways.

[0079] First, when the preset Chinese segmentation granularity parameter is word granularity, the Chinese sequences in the to-be-processed text are word segmented to obtain single Hanzi.

[0080] Second, when the preset Chinese segmentation granularity parameter is character granularity, the Chinese sequences in the to-be-processed text are character segmented to obtain Hanzi combinations.

[0081] The Chinese segmentation granularity parameter is pre-configured, and after the Chinese segmentation granularity parameter is configured, the same granularity is used for Chinese segmentation of each to-be-processed text. The Chinese segmentation granularity parameter can be configured according to an actual application scenario. In an embodiment, the Chinese segmentation granularity parameter is first read from the configuration data. If the read Chinese segmentation granularity parameter is a word granularity, the Chinese sequence in the to-be-processed text is subjected to word segmentation, and a segmentation result of a single Chinese character is obtained. Through word segmentation, fine segmentation of the text can be realized, a larger number of segmented word blocks can be obtained, more dimensional feature data can be obtained, and the accuracy of the classification result can be improved. If the read Chinese segmentation granularity parameter is a word granularity, the Chinese sequence in the to-be-processed text is subjected to word segmentation, and a segmentation result of a Chinese character combination is obtained. Through word segmentation, the integrity of the word meaning can be ensured, meaningless word groups can be avoided, the number of segmented word groups can be simplified, the data processing amount of the model can be reduced, and the processing speed can be improved.

[0082] In an embodiment, taking the identification process of an advertisement comment in a large amount of comment data as an example, considering that the words in the advertisement sentence are mostly words that can be easily identified and screened by the platform, the publisher of the advertisement comment more often replaces one or more words in the original advertisement with homophonic characters or similar characters to avoid being identified and shielded by the platform, and therefore, there are fewer normal words in the published advertisement comment. Based on this application scenario, by configuring the Chinese segmentation granularity parameter as a word granularity, the classification analysis is performed through segmentation of a single Chinese character in the comment, which can better meet the actual needs of the scenario and obtain a more accurate classification result.

[0083] In step 306, the non-Chinese sequence is subjected to subword segmentation to obtain a non-Chinese word block of a preset classification category.

[0084] The preset classification category includes at least one of a mathematical character category, an English character category, a mathematical character-English character combination category, an other character category, and a subword category. Further, the subword category includes at least one of subwords corresponding to the mathematical character category, the English character category, the mathematical character-English character combination category, and the other character category. For example, the commonly used number combination “123” belongs to the mathematical character category, and the number “3” in “x3” belongs to the subword of the mathematical character category. It should be noted that not every segmentation result of a non-Chinese sequence has a subword. For example, after the “get666” is subjected to subword segmentation, the obtained segmented word blocks can be “get” and “666”. The “get” belongs to the English character category, and the “666” belongs to the mathematical character category.

[0085] Through subword segmentation, the non-Chinese word block can be further segmented according to the classification category, the entire classification text can be finely split, and a more fine word block segmentation result can be obtained.

[0086] In one embodiment, as shown in Figure 4 The character sequence in the to-be-processed text is segmented by character type to obtain the segmented word blocks, including steps 402 to 406.

[0087] In step 402, the common character sequence and the uncommon character sequence in the to-be-processed text are identified according to a preset word table containing common characters.

[0088] In step 404, the Chinese sequence in the common character sequence is segmented by word and the non-Chinese sequence in the common character sequence is segmented by sub-word according to the common character category, to obtain the segmented word blocks corresponding to the common character category.

[0089] In step 406, the Chinese sequence in the uncommon character sequence is segmented by word and the non-Chinese sequence in the uncommon character sequence is segmented by sub-word according to the uncommon character category, to obtain the segmented word blocks corresponding to the uncommon character category.

[0090] The common characters refer to the characters in the word table pre-entered based on the usage frequency, including number combinations, letter combinations, number-letter combinations, Chinese, and other common characters. For example, the number combination, i.e., composed of only numbers, such as 123, 666, etc.; the letter combination, i.e., composed of only English letters, such as www, happy, etc.; the number-letter combination, i.e., composed of only numbers and letters, such as 50ml, 128GB, etc.; the Chinese, i.e., a single Chinese character; and other common characters, including Korean, Japanese, and other commonly used characters in other languages.

[0091] Further, the common character category identification carried by the segmented word block corresponds to the common character category. In addition to the categories corresponding to the number combination, letter combination, number-letter combination, Chinese, and other common characters, the common character category also includes the sub-word categories corresponding to the single number, single letter, number-letter combination, Chinese, and other common characters, respectively. For example, a single letter not related to the previous text, etc. In an embodiment, the number combination, letter combination, number-letter combination, Chinese, and other common characters with higher usage frequency can be classified respectively, and the single number, single letter, mathematical symbol, English word affix or punctuation, number-letter combination symbol, Chinese punctuation symbol, etc. with lower usage frequency can be additionally classified as a category.

[0092] The characters included in the preset vocabulary and sub-words are common characters, and the characters not included in the preset vocabulary and sub-words are non-common characters. Correspondingly, a sequence composed of multiple common characters is a common character sequence, and a sequence containing non-common characters is a non-common character sequence. It should be noted that a non-common character sequence does not necessarily consist entirely of non-common characters and can be a sequence containing one or more non-common characters. For example, if the first, third, and last characters in a character sequence are non-common characters and the remaining characters are common characters, then the character sequence is a non-common character sequence.

[0093] In the embodiment, as shown in Table 1, non-common characters include Chinese (UNK_ZH), alphanumeric characters (UNK_ALPHANUMERIC), emoji characters (UNK_EMOJI), and others (UNK), and the corresponding sub-words of the above 4 categories are respectively denoted as ##UNK_ZH, ##UNK_ALPHANUMERIC, ##UNK_EMOJI, ##UNK. Specifically, for non-common Chinese characters in a non-common character sequence, when it appears as the first character in the non-common character sequence, it can be classified as UNK_ZH, and when it appears after other characters, it can be classified as ##UNK_ZH. UNK_ZH and ##UNK_ZH can be used to distinguish their positional relationships with other characters in the non-common character sequence.

[0094] Table 1: Classification categories corresponding to word segmentation blocks

[0095]

[0096] For example, assume the text to be processed is "00X0 · YYM某某某平台". Then the corresponding word segmentation result is: "00 / ##x / ##0 / ## · / ##[UNK_EMOJI] / ##[UNK_EMOJI] / ##m / 某 / 某 / 某 / 平 / 台". Correspondingly, the category identifiers of each word segmentation block are: [num, sub_letters, num, sub_num, ##[UNK_EMOJI, ##[UNK_EMOJI, sub_letter, zh, zh, zh, zh, zh].

[0097] It can be understood that the classification categories corresponding to the word segmentation blocks can be set as needed. In other embodiments, the classification categories corresponding to the word segmentation blocks further include but are not limited to other various forms such as English words, common Chinese characters, rare Chinese characters, and Internet terms. For example, for a pure English category, it can be split into a normal English word category (such as "happy") and a non-English word category (such as "www"); for a Chinese category, it can be split into common Chinese (such as "我") and rare Chinese characters (such as "㧴"), etc.

[0098] In an embodiment, the above steps 202 to 210 can be performed by a text classification model.

[0099] The text classification model refers to a language processing model used for classifying text. For example, a BERT model (Bidirectional Encoder Representation from Transformers) or the like. The text classification model can be trained in advance by sample data carrying classification labels. When the text classification model is used in an application scenario of distinguishing advertisements, the classification labels can be advertisement category labels and non-advertisement category labels. When the text classification model is applied to other application scenarios, the classification labels are category labels suitable for the scenario, such as sensitive category labels and non-sensitive category labels in a sensitive information distinguishing application scenario, or specific software passwords and non-specific software passwords in a specific software password recognition application scenario.

[0100] The number of classification labels can be two, that is, yes or no, and the two are opposite. In other embodiments, the number of classification labels can also be multiple as defined by the user. For example, the advertisement recognition scenario can be further divided into multiple categories of classification labels such as symbolic advertisements, semantic advertisements, and non-advertisements.

[0101] In an embodiment, the text to be processed can be a short text with a relatively small number of characters or a long text with a relatively large number of characters. For example, a text composed of only 10 characters, or a text composed of only one sentence. A long text is, for example, a paragraph or multiple paragraphs composed of multiple sentences.

[0102] In an embodiment, the text classification model is also configured with a Chinese segmentation granularity parameter. The configuration of the Chinese segmentation granularity parameter can be performed before the start of the training process of the text classification model, to ensure that the model training process and the application process of the model are processed based on the same conditions, thereby ensuring the accuracy of the classification results predicted by the model.

[0103] In an embodiment, the text classification model can also be updated in real time to meet the rapid iteration of the model. The model updating method includes: obtaining newly added labeled text carrying a target category label, adding the labeled text to a preset sample library, and recording the number of newly added labeled texts in the preset sample library; when the number of newly added labeled texts reaches a preset threshold, iteratively training the text classification model according to the newly added labeled texts to obtain an updated text classification model.

[0104] In an embodiment, the text classification model comprises a hierarchically compressed BERT model. In one embodiment, the compression process of the BERT model can be implemented by means of knowledge distillation. In other embodiments, the compression process of the BERT model can also be implemented by means of teacher annealing.

[0105] In one embodiment, as shown in Figure 5 The training process of the hierarchically compressed BERT model comprises the following steps 502 to 508.

[0106] Step 502, obtain training data carrying class labels, train the initial BERT model to obtain a teacher model.

[0107] Step 504, predict the training data according to the teacher model to obtain the classification probability of the training data.

[0108] Step 506, hierarchically compress the initial BERT model to obtain a compressed BERT model.

[0109] Step 508, based on a preset loss function, train the compressed BERT model according to the training data carrying the classification probability and the class labels to obtain the BERT model.

[0110] The feature data acquisition process of the initial BERT model comprises word block segmentation, word block classification marking and word block position marking.

[0111] The teacher model refers to the object used for reference of the compressed model, and the model obtained by training is a student model corresponding to the teacher model. The preset loss function is as follows:

[0112]

[0113] Wherein, T refers to the parameter space, t is the tth parameter in the parameter space, D refers to the sample data set, i refers to the ith sample in the sample data set, L refers to the loss function of the entire training data set, y refers to the artificially labeled classification result, and f function represents the output of the teacher model and the output of the student model, respectively.

[0114] In one embodiment, the BERT model comprises Token Embeddings, Token Type Embeddings and position Embeddings, which are used for word block segmentation, word block classification marking and word block position marking, respectively, and are embedded to obtain feature data. Specifically, compared with the original data processing process of the BERT model, the original SegmentEmbeddings is modified to Token Type Embeddings to realize the marking of the classification of the segmented word blocks.

[0115] In an embodiment, the initial BERT model is composed of 12 layers of transformer encoder. Since the model parameters are large, the calculation amount is large, and the prediction time is long, the performance of the initial BERT model is not high. The number of layers of the hierarchical compression BERT model is 3 layers.

[0116] In an embodiment, the feature data acquisition process of the initial BERT model further includes segment division. As shown in Figure 6 According to the segmented token block, the category identifier of the segmented token block, and the position of the segmented token block in the text to be processed, the feature data of the text to be processed further includes steps 602 to 604.

[0117] Step 602, segmenting the text to be processed to determine the segment to which the segmented token block belongs.

[0118] Step 604, according to the segmented token block, the category identifier of the segmented token block, the segment to which the segmented token block belongs, and the position of the segmented token block in the text to be processed, obtaining the feature data of the text to be processed.

[0119] Segment division refers to the process of segmenting the text to be processed, which can be realized by Segment Embeddings in the BERT model. That is, Token Type Embeddings can not replace the original Segment Embeddings, but can be added to the original hierarchical compression BERT model as a new kind of embedding, realizing the expansion of the feature data dimension.

[0120] In an embodiment, the text classification method can be applied to a text management platform. By collecting and synchronizing the labeled text to the sample library in a timely manner, the iterative training of the model is realized, and the online model is updated in a timely manner.

[0121] In an embodiment, as shown in Figure 7 The text classification method further includes steps 702 to 706.

[0122] Step 702, obtaining a new labeled text carrying a target category label.

[0123] Step 704, adding the labeled text to the preset sample library, and recording the number of new labeled texts in the preset sample library.

[0124] Step 706, when the number of new labeled texts reaches a preset threshold, iteratively training the BERT model according to the new labeled texts to obtain an updated BERT model.

[0125] The newly added labeled text includes text in which the BERT model classification result is wrong and is correctly labeled with a correct category. Specifically, the newly added labeled text can be an advertisement sample that is not recognized by the BERT model or a non-advertisement sample that is mistakenly damaged by the model. Specifically, the labeled text can be a sample that is mistakenly damaged and leaked by the model collected by platform managers through quality inspection and the like.

[0126] In an embodiment, taking the target category label as an advertisement label as an example, the uploading interface of the labeled text of the text management platform is as shown in Figure 8 The uploading interface provides two ways of single text uploading and batch uploading. The single text uploading can directly input the text in the text box, select the advertisement category label corresponding to the text, and upload. Specifically, it can be a symbolic advertisement or a semantic advertisement, or a non-advertisement. Batch uploading can be achieved by selecting a file containing multiple newly added labeled texts carrying advertisement category labels, uploading multiple texts in the file by submitting the file, and obtaining the newly added labeled texts carrying advertisement category labels uploaded by the front end in the background of the terminal. The labeled text is added to the preset sample library. The preset sample library refers to a database for storing newly added labeled texts carrying advertisement category labels. By recording the number of newly added labeled texts in the preset sample library, when the number of newly added labeled texts reaches a preset threshold, the BERT model is iteratively trained according to the newly added labeled texts, and an updated BERT model is obtained. When the BERT model is iteratively trained using the newly added labeled texts, the number of newly added labeled texts in the preset sample library is cleared, and the recording of newly added labeled texts is restarted for the next model iterative update. In addition, the text management platform also has an interface that can display existing samples in the sample library and their labeling results, and online call the BERT model to predict whether the input text is an advertisement, so as to facilitate real-time viewing of the BERT model effect.

[0127] In one embodiment, as shown in Figure 9 a text classification method is provided, comprising the following steps:

[0128] Step 902, obtaining training data carrying category labels, training an initial BERT model to obtain a teacher model, and the feature data acquisition process of the initial BERT model includes word block segmentation, word block classification marking, and word block position marking.

[0129] Step 904, predicting the training data according to the teacher model to obtain the classification probability of the training data.

[0130] Step 906, performing hierarchical compression on the initial BERT model to obtain a compressed BERT model.

[0131] Step 908, based on the preset loss function, the compressed BERT model is trained according to the training data carrying the classification probability and the category label, and the BERT model is constructed.

[0132] Step 910, input the text to be processed into the BERT model.

[0133] The following steps 912 to 924 are performed by the BERT model.

[0134] Step 912, according to the preset word table containing common characters, the common character sequence and the uncommon character sequence of the text to be processed are identified.

[0135] Step 914, according to the common character category, the Chinese sequence in the common character sequence is word segmented and the non-Chinese sequence in the common character sequence is sub-word segmented, to obtain the segmented word block corresponding to the common character category.

[0136] Step 916, according to the uncommon character category, the Chinese sequence in the uncommon character sequence is word segmented and the non-Chinese sequence in the uncommon character sequence is sub-word segmented, to obtain the segmented word block corresponding to the uncommon character category.

[0137] Step 918, the position of the segmented word block in the text to be processed is determined, and the category identifier of the segmented word block is obtained according to the category to which the segmented word block belongs.

[0138] Step 920, according to the segmented word block, the category identifier of the segmented word block, and the position of the segmented word block in the text to be processed, the feature data of the text to be processed is obtained.

[0139] Step 922, according to the feature data, the text to be processed is classified and analyzed to obtain a classification result.

[0140] Step 924, obtain the newly added labeled text carrying the advertisement category label.

[0141] Step 926, add the labeled text to the preset sample library, and record the number of newly added labeled texts in the preset sample library.

[0142] Step 928, when the number of newly added labeled texts reaches a preset threshold, the BERT model is iteratively trained according to the newly added labeled texts to obtain an updated BERT model.

[0143] The application also provides an application scenario of the advertisement text recognition, which applies the above-mentioned text classification method. Specifically, the application of the text classification method in the application scenario is as follows:

[0144] Firstly, a 12-layer BERT model is obtained, and a knowledge distillation method is used to compress the 12-layer model into a 3-layer model, and the hierarchical compressed BERT model is trained. First, train the 12-layer BERT model as a teacher model, then use the teacher model to predict the training data to obtain the probability output of each training data, based on the preset loss function, according to the classification probability output by the teacher model and the classification label carried by the training data, the compressed BERT model is trained to obtain the BERT model used for advertising text recognition.

[0145] Among them, compared with the traditional BERT model, the BERT model in the scheme has the following improvements in structure:

[0146] (1) Modify WordPiece Tokenizer: Further word segmentation for very rare text sequence UNK.

[0147] The BERT model is word granularity for Chinese segmentation, and the Chinese is cut word by word, and the other non-Chinese fragments are segmented into subwords, that is, cut into subwords. In the traditional segmentation process, once the non-Chinese sequence contains characters that do not appear in the word table, the entire sequence will be treated as UNK. Apply the characters of such very rare text sequences, continue to do subword segmentation, and process the non-appearance characters as "##UNK".

[0148] (2) Increase Token Type Embeddings: classify each token block (token), and add the category information as a feature in the form of embedding to the BERT model.

[0149] Among them, the classification rules of the token block include: for non-UNK words (i.e. subwords and common words contained in the word table), according to whether the token is a number, a letter, etc. With suspected advertising features, it is divided into specific, including number combination, letter combination, number-letter combination, Chinese and other common characters. For UNK words, in addition to Chinese, number-letter combination, other characters, add the EMOJI category.

[0150] As shown in Figure 10 From the structure of the BERT model, the SegmentEmbeddings used for segment division in the model is modified to Token Type Embeddings used for category labeling. Taking the input text "00X0 · YYM someone platform" as an example, the input text, the word segmentation result and Token Type Embeddings are respectively:

[0151] Input text: 00X0 · YYM someone platform;

[0152] Word segmentation result: 00 / ##x / ##0 / ## · / ##[UNK_EMOJI] / ##[UNK_EMOJI] / ##m / certain / certain / certain / flat / platform;

[0153] Category label: [num,sub_letters,num,sub_num,##[UNK_EMOJI, ##[UNK_EMOJI,sub_letter, zh, zh, zh, zh, zh];

[0154] According to the word segmentation blocks, category identifiers, and the positions of the word segmentation blocks in the text to be processed, the feature data of the text to be processed is obtained; finally, the BERT model performs classification analysis on the text to be processed based on the feature data to obtain the classification result.

[0155] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0156] In one embodiment, as Figure 11 shown, a text classification device 1100 is provided. This device can adopt software modules or hardware modules, or a combination of both to become a part of a computer device. Specifically, the device includes: a text acquisition module 1102, a word block segmentation module 1104, a category and position determination module 1106, a feature data processing module 及分类分析模块1110, where:

[0157] The text acquisition module 1102 is used to acquire the text to be processed.

[0158] The word block segmentation module 1104 is used to perform segmentation processing on the character sequence in the text to be processed according to the character type to obtain the segmented word blocks.

[0159] The category and position determination module 1106 is used to determine the position of the segmented word block in the text to be processed and obtain the category identifier of the segmented word block according to the category to which the segmented word block belongs.

[0160] The feature data processing module 1108 is configured to obtain feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks, and the positions of the segmented word blocks in the text to be processed.

[0161] The classification analysis module 1110 is configured to perform classification analysis on the text to be processed according to the feature data, and obtain a classification result.

[0162] In an embodiment, the word block segmentation module is further configured to identify Chinese sequences and non-Chinese sequences in the text to be processed, segment the Chinese sequences according to a preset Chinese segmentation granularity parameter to obtain Chinese word blocks, and perform sub-word segmentation on the non-Chinese sequences to obtain non-Chinese word blocks of a preset classification category, wherein the preset classification category includes at least one of a mathematical character category, an English character category, a mathematical character-English character combination category, an other character category, and a sub-word category.

[0163] In an embodiment, the word block segmentation module is further configured to perform word segmentation on the Chinese sequences in the text to be processed to obtain single Chinese characters when the preset Chinese segmentation granularity parameter is a character granularity, and perform word segmentation on the Chinese sequences in the text to be processed to obtain Chinese character combinations when the preset Chinese segmentation granularity parameter is a word granularity.

[0164] In an embodiment, the word block segmentation module is further configured to identify common character sequences and uncommon character sequences in the text to be processed according to a preset word table containing common characters, perform word segmentation on the Chinese sequences in the common character sequences and sub-word segmentation on the non-Chinese sequences in the common character sequences according to a common character category to obtain segmented word blocks corresponding to the common character category, and perform word segmentation on the Chinese sequences in the uncommon character sequences and sub-word segmentation on the non-Chinese sequences in the uncommon character sequences according to an uncommon character category to obtain segmented word blocks corresponding to the uncommon character category.

[0165] In an embodiment, the text classification device includes a BERT model for implementing the text classification method in any of the above embodiments, and further includes a model training module. The model training module is configured to obtain training data carrying category labels, train an initial BERT model to obtain a teacher model, and obtain feature data of the initial BERT model through word block segmentation, word block category labeling, and word block position labeling. The model training module is further configured to predict the training data according to the teacher model to obtain classification probabilities of the training data, perform hierarchical compression on the initial BERT model to obtain a compressed BERT model, and train the compressed BERT model based on a preset loss function and according to the training data carrying the classification probabilities and the category labels to construct the BERT model.

[0166] In an embodiment, the text classification apparatus further comprises a model updating module configured to: acquire newly-labeled text carrying an advertising category label, the newly-labeled text comprising text whose BERT model classification result is incorrect and for which a correct category label is added; add the labeled text to a preset sample library and record the number of newly-labeled texts in the preset sample library; and when the number of newly-labeled texts reaches a preset threshold, iteratively train the BERT model according to the newly-labeled texts to obtain an updated BERT model.

[0167] In an embodiment, the feature data processing module is further configured to: divide the text to be processed into segments and determine the segment to which each segmented word block belongs; and obtain the feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks, the segments to which the segmented word blocks belong, and the positions of the segmented word blocks in the text to be processed.

[0168] The above text classification method comprises: acquiring a text to be processed; performing segmentation processing on character sequences in the text to be processed according to character types to obtain segmented word blocks, thereby achieving fine segmentation of characters in the text to be processed; determining the positions of the segmented word blocks in the text to be processed, and obtaining category identifiers of the segmented word blocks according to the categories to which the segmented word blocks belong; and obtaining feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks, and the positions of the segmented word blocks in the text to be processed. The category information of the characters is utilized to obtain feature data with richer semantic knowledge, and the text to be processed is classified and analyzed to obtain a more accurate classification result.

[0169] The specific limitations of the text classification apparatus can be seen from the limitations of the text classification method described above, and will not be described here. Each module in the above text classification apparatus can be implemented in whole or in part by software, hardware, and combinations thereof. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0170] In an embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 12As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a text classification method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0171] Those skilled in the art can understand that, Figure 12 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0172] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0173] In one embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the steps in the above method embodiments.

[0174] In one embodiment, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above method embodiments.

[0175] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, the RAM can be in a variety of forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0176] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0177] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method of text classification, characterized by, The method comprises: acquiring a text to be processed; identifying Chinese sequences and non-Chinese sequences in the text to be processed through a level for word block segmentation in a BERT model, identifying common character sequences and uncommon character sequences of the text to be processed according to a preset word table and subwords, the characters contained in the preset word table and subwords being common characters, performing word segmentation on the Chinese sequences in the common character sequences according to a common character category and performing subword segmentation on the non-Chinese sequences in the common character sequences to obtain segmented word blocks corresponding to the common character category, and performing word segmentation on the Chinese sequences in the uncommon character sequences according to an uncommon character category and performing subword segmentation on the non-Chinese sequences in the uncommon character sequences to obtain segmented word blocks corresponding to the uncommon character category; the uncommon character category includes a category of uncommon characters and a category of subwords corresponding to the category of uncommon characters; the uncommon character category increases an emoticon character category compared with the common character category; obtaining category identifiers of the segmented word blocks according to categories to which the segmented word blocks belong through a level for word block category marking in the BERT model, wherein when the segmented word blocks include special characters, the special characters are subjected to escape processing according to category identifiers of adjacent characters of the special characters; the category identifiers include respective identifiers of the common character category and the uncommon character category; determining positions of the segmented word blocks in the text to be processed through a level for word block position marking in the BERT model; obtaining feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks and the positions of the segmented word blocks in the text to be processed; performing classification analysis on the text to be processed according to the feature data to obtain a classification result; the classification result indicates whether the text to be processed is an advertisement.

2. The method of claim 1, wherein, The common character category includes a mathematical character category, an English character category, a mathematical character-English character combination category, an other character category and a subword category.

3. The method of claim 2, wherein, The Chinese sequences are subjected to word segmentation at a preset Chinese segmentation granularity parameter of word granularity.

4. The method of claim 1, wherein, The obtaining of the feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks and the positions of the segmented word blocks in the text to be processed comprises: performing segment division on the text to be processed to determine segments to which the segmented word blocks belong; obtaining the feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks, the segments to which the segmented word blocks belong and the positions of the segmented word blocks in the text to be processed.

5. A method of text classification characterized by, The BERT model is used to implement the text classification method according to any one of claims 1-4; The construction process of the BERT model comprises: acquiring training data carrying category labels, training an initial BERT model to obtain a teacher model, and the feature data acquisition process of the initial BERT model comprises word block segmentation, word block category marking and word block position marking; According to the teacher model, the training data is predicted to obtain classification probability of the training data; The initial BERT model is compressed to obtain a compressed BERT model; Based on a preset loss function, the compressed BERT model is trained according to the training data carrying the classification probability and the category label, and the BERT model is constructed.

6. The method of claim 5, wherein, The method further comprises: An additional labeled text carrying a target category label is obtained, and the additional labeled text includes a text for which the BERT model classification result is incorrect and for which a correct category label is added; The labeled text is added to a preset sample library, and the number of additional labeled texts in the preset sample library is recorded; When the number of additional labeled texts reaches a preset threshold, the BERT model is iteratively trained according to the additional labeled texts to obtain an updated BERT model.

7. A text classification apparatus characterized by comprising: The device comprises: A text acquisition module is configured to acquire a text to be processed; A word block segmentation module is configured to identify Chinese sequences and non-Chinese sequences in the text to be processed through a layer for word block segmentation in the BERT model, identify common character sequences and uncommon character sequences in the text to be processed according to a preset word table and subwords, identify Chinese sequences in the common character sequences according to a common character category and perform subword segmentation on non-Chinese sequences in the common character sequences to obtain segmented word blocks corresponding to the common character category, and identify Chinese sequences in the uncommon character sequences according to an uncommon character category and perform subword segmentation on non-Chinese sequences in the uncommon character sequences to obtain segmented word blocks corresponding to the uncommon character category; the uncommon character category includes a category of uncommon characters and a category of subwords corresponding to the category of uncommon characters; the uncommon character category includes a category of emoticon characters in addition to the common character category; A category and position determination module is configured to obtain category identifiers of the segmented word blocks according to categories to which the segmented word blocks belong through a layer for word block category marking in the BERT model, and when the segmented word blocks include special characters, to perform escape processing on the special characters according to category identifiers of adjacent characters of the special characters; the category identifiers include respective identifiers of the common character category and the uncommon character category; and to determine positions of the segmented word blocks in the text to be processed through a layer for word block position marking in the BERT model. A feature data processing module is configured to obtain feature data of the text to be processed according to the segmented word blocks, the category identifiers of the segmented word blocks, and the positions of the segmented word blocks in the text to be processed. A classification analysis module is configured to perform classification analysis on the text to be processed according to the feature data to obtain a classification result; the classification result indicates whether the text to be processed is an advertisement.

8. The text classification apparatus according to claim 7, characterized by The common character category includes a mathematical character category, an English character category, a mathematical character-English character combination category, an other character category, and a subword category.

9. The text classification apparatus according to claim 8, characterized by The Chinese sequence is cut by word granularity, and the Chinese cutting granularity parameter is preset as word granularity.

10. The text classification apparatus according to claim 7, characterized by The feature data processing module is further configured to divide the to-be-processed text into segments, determine a segment to which the cut word block belongs, and obtain feature data of the to-be-processed text according to the cut word block, the category identifier of the cut word block, the segment to which the cut word block belongs, and a position of the cut word block in the to-be-processed text.

11. A text classification apparatus characterized by comprising: The apparatus comprises: a training data acquisition module configured to acquire training data carrying category labels; a teacher model training module configured to train an initial BERT model to obtain a teacher model, wherein a feature data acquisition process of the initial BERT model comprises word block cutting, word block category labeling, and word block position labeling; a probability prediction module configured to predict the training data according to the teacher model to obtain category probabilities of the training data; a model compression module configured to compress the initial BERT model by levels to obtain a compressed BERT model; a model training and construction module configured to train the compressed BERT model based on a preset loss function and according to the training data carrying the category probabilities and the category labels, and to construct a BERT model, wherein the BERT model is configured to implement the text classification method according to any one of claims 1-4.

12. The text classification apparatus according to claim 11, characterized by The apparatus further comprises: an added labeled text acquisition module configured to acquire added labeled text carrying target category labels, wherein the added labeled text comprises text whose classification result by the BERT model is incorrect and for which correct category labeling is performed; a sample library updating module configured to add the labeled text to a preset sample library and record a number of added labeled texts of the preset sample library; a model iterative training module configured to, when the number of added labeled texts reaches a preset threshold, iteratively train the BERT model according to the added labeled texts to obtain an updated BERT model.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the method of any one of claims 1-6 when executing the computer program.

14. A computer readable storage medium storing a computer program, wherein the computer program comprises program instructions configured to cause a processor to perform the method according to any one of claims 1 to 13. The computer program implements the method of any one of claims 1-6 when executed by the processor.

15. A computer program product comprising computer instructions, characterized in that, The computer program implements the method of any one of claims 1-6 when executed by the processor. The computer program implements the method of any one of claims 1-6 when executed by the processor.

Images