Classification model training method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a classification model training method and device, electronic equipment and storage medium, which are applied to the technical field of model training. The classification model training method comprises: obtaining a plurality of target texts; obtaining a classification prompt corresponding to a preset category; inputting the classification prompt and the target text into a text labeling model to obtain an output result; taking a target text of a target category as the preset category and the preset category as a group of training samples to obtain a plurality of groups of training samples; inputting the training samples into an initial classification model to obtain a predicted classification result corresponding to the training samples; calculating a loss value between the predicted classification result and the preset category of the training sample, and updating an increment matrix by using the loss value to obtain a classification model. The classification model obtained by using the classification model training method provided by the embodiments of the present application can save the calculation resources required for text classification.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and in particular to a classification model training method, apparatus, electronic device, and storage medium. Background Technology

[0002] When browsing news or watching videos, people often express their opinions on the content, usually through comments or bullet screens. However, influenced by subjective biases, not all opinions are objective or positive; some convey negative values ​​or unfriendly concepts. The presence of these negative opinions can negatively impact the user's browsing or viewing experience.

[0003] Therefore, in related technologies, user-expressed opinions are usually combined with classification prompts and closed-source large language models, such as GPT-4 (Generative Pre-trained Transformer 4), to classify these opinions, identify negative opinions, and avoid displaying them, thereby improving the user's browsing or viewing experience.

[0004] However, the classification methods of related technologies require lengthy classification prompts to achieve accurate classification, which results in a significant consumption of computing resources when classifying real-time and massive amounts of viewpoints. Summary of the Invention

[0005] The purpose of this invention is to provide a classification model training method, apparatus, electronic device, and storage medium to save computational resources when classifying text. The specific technical solution is as follows:

[0006] In a first aspect of this invention, a classification model training method is provided, the method comprising:

[0007] Acquire multiple target texts; the multiple target texts include preset keywords belonging to preset categories;

[0008] Obtain the category suggestion corresponding to the preset category;

[0009] The classification hints and the target text are input into the text annotation model to obtain the output of the text annotation model; wherein, the text annotation model predicts and labels the target category of each target text based on the input classification hints and the target text;

[0010] In the output results, a target text whose target category is the preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples;

[0011] The following steps are repeated with the selected training samples: for any selected set of training samples, the training samples are input into the initial classification model to obtain the predicted classification result corresponding to the training samples; wherein, the weight matrix in the initial classification model is obtained by superimposing a general weight matrix and an increment matrix; the increment matrix corresponds to the category of the multiple training samples;

[0012] Calculate the loss value between the predicted classification result and the preset categories included in the training samples, update the increment matrix using the loss value, and continue until the termination condition is met to obtain the classification model.

[0013] Optionally, obtaining multiple target texts includes:

[0014] Retrieve multiple raw texts;

[0015] Using preset keywords, the multiple original texts are filtered, and the original texts containing the preset keywords are retained as target texts; the preset keywords are keywords of preset categories.

[0016] Optionally, the process by which the text annotation model predicts and annotates the target category of each target text based on the input classification prompts and the target text includes:

[0017] The classification prompts and the target text are segmented into words to obtain a segmentation set;

[0018] Using the word vectors corresponding to each word in the word segmentation set, a target matrix corresponding to the classification prompt and the target text is generated;

[0019] Predict the confidence level of the target text belonging to each category based on the target matrix;

[0020] The category with the highest confidence level is taken as the target category of the target text, and the target category is labeled accordingly.

[0021] Optionally, before inputting the classification prompts and the target text into the text annotation model, the method further includes:

[0022] For each target text, identify the names of people in the target text;

[0023] Query the gender of the person indicated by the name, and replace the name of the person in the target text with a preset identifier corresponding to the gender;

[0024] The preset category includes multiple subcategories; the multiple subcategories include a subcategory corresponding to the character's gender; the classification prompts include multiple sub-classification prompts corresponding to each of the multiple subcategories; the multiple sub-classification prompts include a sub-classification prompt for the subcategory corresponding to the character's gender; the classification prompts also include the correspondence between the preset identifier and the character's gender; the step of inputting the classification prompts and the target text into the text annotation model includes:

[0025] The classification prompts and the replaced target text are input into the text annotation model; the text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the person's gender based on the correspondence between each subcategory prompt, the preset identifier and the person's gender. If so, the subcategory corresponding to the person's gender is used as the target category of the target text for annotation.

[0026] Optionally, before inputting the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples, the method further includes:

[0027] Create an increment matrix corresponding to the preset categories to which multiple sets of training samples belong; the increment matrix is ​​initially a zero matrix.

[0028] The weight matrix of the initial classification model is obtained by superimposing a general weight matrix with the incremental matrix.

[0029] Optionally,

[0030] The acquisition of multiple target texts includes:

[0031] Multiple target bullet comments are obtained; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, and the bullet comments include general negative keywords.

[0032] Obtaining category suggestions corresponding to the preset category includes:

[0033] Obtain negative semantic category classification suggestions; the negative semantic category classification suggestions include multiple negative semantic subcategories and sub-classification suggestions for each negative semantic subcategory;

[0034] The step of inputting the classification prompts and the target text into the text annotation model to obtain the output of the text annotation model includes:

[0035] The negative semantic category classification prompt and the multiple target bullet comments are input into the GPT language model to obtain a form output by the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comments based on the negative semantic category classification prompt and the target bullet comments; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment.

[0036] In the output result, a target text whose target category is the preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples, including:

[0037] In the form, a target barrage whose target subcategory is the negative semantic subcategory is selected, and the negative semantic subcategory of the target barrage is used as a set of training samples to obtain multiple sets of training samples.

[0038] In a second aspect of the invention, a classification model training apparatus is provided, the apparatus comprising:

[0039] The first acquisition module is used to acquire multiple target texts; the multiple target texts include preset keywords belonging to preset categories;

[0040] The second acquisition module is used to acquire classification prompts corresponding to the preset category;

[0041] The annotation module is used to input the classification prompts and the target text into the text annotation model and obtain the output results of the text annotation model; wherein, the text annotation model predicts and annotates the target category of each target text based on the input classification prompts and the target text;

[0042] The third acquisition module is used to take a target text whose target category is the preset category and the preset category as a set of training samples in the output result to obtain multiple sets of training samples.

[0043] The prediction module is used to select the training samples and repeatedly perform the following steps: for any selected set of training samples, input the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples; wherein, the weight matrix in the initial classification model is obtained by superimposing a general weight matrix and an increment matrix; the increment matrix corresponds to the category of the multiple training samples;

[0044] The update module is used to calculate the loss value between the predicted classification result and the preset categories included in the training samples, and update the increment matrix using the loss value until the termination condition is met to obtain the classification model.

[0045] Optionally, the first acquisition module is specifically used to acquire multiple original texts; filter the multiple original texts using preset keywords, and retain the original texts including the preset keywords as target texts; the preset keywords are keywords of preset categories.

[0046] Optionally, the text annotation model is specifically used to segment the classification prompt and the target text into words to obtain a word segmentation set; using the word vector corresponding to each word segment in the word segmentation set, a target matrix corresponding to the classification prompt and the target text is generated; the confidence level of the target text belonging to each category is predicted based on the target matrix; the category with the highest confidence level is taken as the target category of the target text, and the target category is labeled accordingly for the target text.

[0047] Optionally, the device further includes: a replacement module, configured to, before inputting the classification prompts and the target text into the text annotation model, identify the person's name in each target text; query the gender of the person indicated by the person's name; and replace the person's name in the target text with a preset identifier corresponding to the person's gender;

[0048] The preset category includes multiple subcategories; among the multiple subcategories is a subcategory corresponding to the character's gender; the classification prompts include multiple sub-classification prompts corresponding to each of the multiple subcategories; among the multiple sub-classification prompts is a sub-classification prompt for the subcategory corresponding to the character's gender; the classification prompts also include the correspondence between the preset identifier and the character's gender;

[0049] The annotation module is specifically used to input the classification prompts and the replaced target text into the text annotation model; the text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the person's gender based on the correspondence between each subcategory prompt, the preset identifier and the person's gender, and if so, to annotate the subcategory corresponding to the person's gender as the target category of the target text.

[0050] Optionally, the device further includes:

[0051] A creation module is used to create an increment matrix corresponding to a preset category of multiple training samples before inputting the training samples into the initial classification model and obtaining the predicted classification result corresponding to the training samples; the increment matrix is ​​initially a zero matrix; a general weight matrix is ​​superimposed on the increment matrix to obtain the weight matrix of the initial classification model.

[0052] Optionally,

[0053] The first acquisition module is specifically used to acquire multiple target bullet comments; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, and the bullet comments include general negative keywords.

[0054] The second acquisition module is specifically used to acquire negative semantic category classification prompts; the negative semantic category classification prompts include multiple negative semantic subcategories and sub-classification prompts for each negative semantic subcategory;

[0055] The annotation module is specifically used to input the negative semantic category classification prompts and the multiple target bullet comments into the GPT language model to obtain a form output by the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comments based on the negative semantic category classification prompts and the target bullet comments; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment.

[0056] The third acquisition module is specifically used to select a target barrage whose target subcategory is the negative semantic subcategory from the form, and the negative semantic subcategory of the target barrage as a set of training samples to obtain multiple sets of training samples.

[0057] In a third aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0058] The memory is used to store computer programs;

[0059] When the processor executes the program stored in the memory, it implements the steps of the method described in the first aspect above.

[0060] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect above.

[0061] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of the method described in the first aspect.

[0062] The classification model training method provided in this invention only uses a text annotation model to annotate target text of a preset category, avoiding its participation in the classification process of a large number of texts, thus saving computational resources. Training samples are determined based on the output of the text annotation model, and the initial classification model is further trained. The incremental matrix in the initial classification model is gradually updated to obtain a self-developed classification model capable of accurately classifying the preset categories. When using this classification model for subsequent classification, there is no need to input lengthy classification prompts to classify the input text, avoiding the model repeatedly analyzing classification prompts, thereby saving computational resources required for text classification. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0064] Figure 1 A flowchart of a classification model training method provided in an embodiment of the present invention;

[0065] Figure 2 A flowchart of the classification method provided in the embodiments of the present invention;

[0066] Figure 3 This is a schematic diagram of the structure of the classification model training device provided in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0069] When browsing news or watching videos, people often express their opinions on the content, usually through comments or bullet screens. However, influenced by subjective biases, not all opinions are objective or positive; some convey negative values ​​or unfriendly concepts. The presence of these negative opinions can negatively impact the user's browsing or viewing experience.

[0070] For example, negative opinions in video comments, such as "lawyers always speak for bad people," "money can buy anything," and "A is not a good girl either," not only affect the atmosphere of the comment community but also easily trigger user reports, which greatly impacts the user's viewing experience.

[0071] In related technologies, user-posted opinions are typically categorized using closed-source large language models, such as GPT-4 (Generative Pre-trained Transformer 4), by combining classification prompts and identifying negative opinions to avoid displaying them, thereby improving the user's browsing or viewing experience.

[0072] However, this classification method requires GPT-4 to input lengthy classification prompts to achieve accurate classification, resulting in significant computational resource consumption when classifying large volumes of real-time generated viewpoints. To address this issue, such as... Figure 1 As shown, this embodiment of the invention provides a classification model training method, the method comprising:

[0073] S101, acquire multiple target texts;

[0074] Among them, multiple target texts include preset keywords belonging to preset categories.

[0075] S102, obtain the category suggestion corresponding to the preset category.

[0076] S103, input the classification prompts and target text into the text annotation model to obtain the output results of the text annotation model;

[0077] The text annotation model predicts and annotates the target category of each target text based on the input classification prompts and target text.

[0078] S104, in the output results, a target text with a target category of a preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples.

[0079] S105, Select training samples and repeat the following steps: For any selected set of training samples, input the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples;

[0080] In the initial classification model, the weight matrix is ​​obtained by superimposing the general weight matrix and the increment matrix; the increment matrix corresponds to the category of multiple training samples.

[0081] S106, calculate the loss value between the predicted classification result and the preset categories included in the training samples, update the increment matrix using the loss value, until the termination condition is met, and obtain the classification model.

[0082] The classification model training method provided in this invention only uses a text annotation model to annotate target text of a preset category, avoiding its participation in the classification process of a large number of texts, thus saving computational resources. Training samples are determined based on the output of the text annotation model, and the initial classification model is further trained. The incremental matrix in the initial classification model is gradually updated to obtain a self-developed classification model capable of accurately classifying the preset categories. When using this classification model for subsequent classification, there is no need to input lengthy classification prompts to classify the input text, avoiding the model repeatedly analyzing classification prompts, thereby saving computational resources required for text classification.

[0083] Reference Figure 1 This invention provides a detailed description of the classification model training method. The classification model training method provided in this invention can include two processes: generating training samples and training the classification model using the training samples. Specifically, the process of generating training samples corresponds to steps S101-S104 in this invention embodiment; the process of training the classification model using the training samples corresponds to steps S105-S106 in this invention embodiment.

[0084] First, the process of generating training samples will be explained.

[0085] S101, acquire multiple target texts;

[0086] Among them, multiple target texts include preset keywords belonging to preset categories.

[0087] When generating training samples, a text annotation model is used to label the categories of each input text, and then texts belonging to the preset categories are selected to construct training samples, thereby obtaining training samples of the preset categories in a targeted manner.

[0088] Therefore, if all the text to be annotated is regarded as the original text, then the annotation process of the text annotation model of the present invention for the original text that is not of the preset category is actually unnecessary.

[0089] Furthermore, due to the sheer volume of original text—for example, the number of bullet comments in a single TV series can reach millions—annotating all the original text (i.e., all the bullet comments for that series) would typically be computationally expensive. Therefore, reasonably reducing the amount of original text to be annotated can save computational costs. For instance, original text that only includes punctuation and repeated interjections does not need to be annotated.

[0090] Therefore, in order to focus the annotation process of the text annotation model as much as possible on the annotation of the original text of the preset category and save the annotation cost of the text annotation model, the original text can be filtered before providing the text annotation model with the original text to be annotated, so as to obtain the target text to be input into the text annotation model.

[0091] After obtaining multiple raw texts, preset keywords can be used to filter them, retaining only the raw texts containing the preset keywords as the target text. The preset keywords are keywords belonging to preset categories.

[0092] Preset keywords can be high-frequency words in a preset category, that is, words that appear frequently in text within that preset category. In this embodiment of the invention, preset keywords can be stored using a preset vocabulary list. The preset vocabulary list can correspond to different categories, and each preset vocabulary list can include multiple preset keywords. For example, the preset category corresponding to the preset vocabulary list might be a negative semantic category, and the categorized words included in this negative semantic category could include "bad," "not good," "get lost," "affected," etc. This embodiment of the invention does not limit the categorized words.

[0093] Preset keywords can be collected manually, and preset thesaurus can also be manually constructed and updated in real time. Technical personnel can pre-select high-frequency characters / words belonging to preset categories from a large amount of raw text as preset keywords for those categories.

[0094] For example, when the preset category is negative semantic category, 10 TV series with high frequency of negative semantic category bullet comments can be selected. The original text is obtained from the bullet comment sets of these TV series, and the negative semantic words / phrases that frequently appear in the bullet comment sets are selected as the preset keywords of negative semantic category.

[0095] In addition, to maintain the timeliness of preset keywords, they can be updated periodically using newly generated raw text. This allows for more accurate retention of raw text belonging to preset categories when filtering multiple raw texts using preset keywords.

[0096] Filtering multiple original texts using preset keywords can include matching each original text with the preset keywords. Since the preset keywords are high-frequency words in a preset category, if an original text contains a preset keyword, it is highly likely that the original text belongs to the preset category. In this case, the original text is considered to meet the filtering conditions and is retained as the target text.

[0097] For example, if the preset keywords include keywords indicating negative semantic categories such as "ugly," "bad person," and "trash," and the obtained original text 1 is "He looks really ugly," and original text 2 is "This person is a good person," then when filtering the original text using the preset keywords, original text 1, containing the preset keyword "ugly," is considered to meet the filtering conditions and is likely to belong to the negative semantic category, and is retained as the target text. Original text 2, however, does not contain any of the aforementioned preset keywords, and therefore is considered not to meet the filtering conditions and is likely not to belong to the negative semantic category, and is not retained.

[0098] Filtering multiple original texts using preset keywords can be considered a coarse filtering process for the categories to which the original texts belong. This ensures that as many of the target texts obtained after coarse segmentation as possible belong to the preset categories, thereby reducing the overhead of subsequent text annotation models in labeling the target texts.

[0099] S102, obtain the category suggestion corresponding to the preset category.

[0100] Since S101 only performs coarse filtering on the original text, it cannot guarantee that all target texts belong to the preset category. Therefore, this embodiment of the invention uses a text annotation model to further filter the target text to obtain the accurate result of the category to which each target text belongs.

[0101] Different classification prompts can correspond to different categories, enabling the text annotation model to classify and label the target text according to the input classification prompts. This embodiment of the invention can obtain classification prompts corresponding to preset categories, thereby enabling the text annotation model to classify and label the target text according to the preset categories.

[0102] In one possible implementation, the classification hints for the preset categories can indicate the major category of the preset category. This allows the text annotation model to determine and label whether the target text belongs to the major category of the preset category. The classification hints for the preset categories can also indicate the subcategories under the preset category, that is, the sub-categories of the preset category, allowing the text annotation model to determine and label which specific sub-category of the preset category the target text belongs to.

[0103] For example, the major category of the preset category can be a negative semantic category, and the minor categories of the preset category can include negative semantic subcategories targeting men, negative semantic subcategories targeting women, etc. If the category of a target text is a negative semantic subcategory targeting women, then when the obtained classification hint indicates the major category of the preset category, the text annotation model will label the target text as a negative semantic category; only when the obtained classification hint indicates a minor category under the preset category will the text annotation model label the target text as a negative semantic subcategory targeting women.

[0104] The classification hints can be pre-edited and stored, or they can be edited in real time according to actual needs. The specific type of classification hints obtained can be selected according to actual needs, and this embodiment of the invention does not impose any restrictions. Generally speaking, in order to more accurately determine the category of the target text, improve the quality of training samples, and enable the initial classification model to achieve more accurate classification after training, classification hints indicating subcategories under a preset category can be obtained. This allows the text annotation model to accurately determine the subcategory of the preset category to which the target text belongs.

[0105] S103, input the classification prompts and target text into the text annotation model to obtain the output results of the text annotation model;

[0106] The text annotation model predicts and annotates the target category of each target text based on the input classification prompts and target text.

[0107] The text annotation model can be a generative large language model, which uses the input content to generate output results that meet the requirements. In the embodiments of this application, classification prompts can also be understood as classification instructions, guiding the text annotation model to generate output results that meet the classification prompts.

[0108] The text annotation model can be an existing, pre-trained model, such as the closed-source large language model GPT-4. In this embodiment of the invention, the target text and classification hints are input into the pre-trained text annotation model, and the text classification model predicts the category to which the target text belongs based on the input classification hints and the target text.

[0109] The text annotation model predicts and annotates the target category of each target text based on the input classification prompts and target text. This process may include: segmenting the classification prompts and target text into words to obtain a word segmentation set; using the word vectors corresponding to each word segment in the word segmentation set to generate a target matrix corresponding to the classification prompts and target text; predicting the confidence level of the target text belonging to each category based on the target matrix; and taking the category with the highest confidence level as the target category of the target text and annotating the target text with the target category.

[0110] The text annotation model learns the semantics of the input classification prompts and target text. By splitting the classification prompts and target text into characters / words, it obtains a word segmentation set corresponding to the input classification prompts and target text. For each word segmentation set, a corresponding word vector is generated. Then, using the vectors corresponding to each word segmentation, a target matrix corresponding to the classification prompts and target text is obtained. This target matrix is ​​then used to predict the category of the target text.

[0111] Depending on the different classification prompts provided in the input, the text annotation model will predict the category of the target text differently.

[0112] In one possible implementation, when the classification hint obtained in S102 indicates a broad category of a preset category, the prediction result can include both the preset category and non-preset categories. For example, when the obtained classification hint indicates a negative semantic category, the prediction result of the target text can include both negative and non-negative semantic categories, that is, the target text can be simply divided into negative and non-negative semantic categories.

[0113] In another possible implementation, when the classification hints obtained in S102 indicate the subcategories of a preset category, the prediction result can include the subcategories and other categories. These other categories can include non-preset categories, or they can include subcategories within the preset category that do not belong to the subcategories included in the classification hints.

[0114] When predicting the category of target text, the text annotation model determines the confidence level of the target text belonging to each of the possible categories (subcategories) included in the prediction results. Taking the classification hint indicating a predefined category obtained in S102 as an example, the text annotation model will determine the confidence level of the target text belonging to the predefined category and non-predefined categories respectively. Then, it selects the category (subcategory) with the highest confidence level as the target category of the target text.

[0115] In one possible implementation, if the category (subcategory) indicated by the classification prompt obtained in S102 is related to gender, for example, if the classification prompt indicates the negative semantic category of the unfriendly female subcategory, then for target texts pointing to different genders, such as "These women only know how to waste money", that is, target text pointing to women, "Zhang San (a certain man) only knows how to waste money", that is, target text pointing to men.

[0116] However, text annotation models often struggle to distinguish the gender indicated by the names of people in the target text. For example, a text annotation model may not know that "Zhang San" refers to a male, which may lead to misclassification or omission of the target text's category. For instance, a target text that actually refers to a male may be classified as a female-unfriendly subcategory.

[0117] Therefore, if the text annotation model can accurately determine the gender of the target text, it can improve the classification accuracy of the text annotation model, thereby obtaining higher quality training samples. In this embodiment of the invention, before inputting the target text into the text annotation model, the names of people appearing in the target text can be replaced, so that the text annotation model can better distinguish the gender indicated by different names, thereby improving the accuracy of determining the category to which the target text belongs.

[0118] In this embodiment of the invention, before inputting the classification prompts and target text into the text annotation model, the method further includes: for each target text, identifying the name of a person in the target text; querying the gender of the person indicated by the name of the person, and replacing the name of the person in the target text with a preset identifier corresponding to the gender of the person.

[0119] A mapping relationship between character names and genders can be pre-established, or a table can be used to store the association between character names and genders. Characters can include real people, such as popular figures, or fictional characters, such as characters appearing in movies and TV series. That is, character names can include both real person names and character names. Character genders can be simply divided into male and female categories. In this embodiment of the invention, the mapping relationship between character names and genders can be continuously updated to ensure its timeliness.

[0120] For each target text, the system identifies whether it contains a character name included in the mapping relationship. For identified character names, based on the mapping relationship between character names and genders, the corresponding gender can be queried, and the character name can be replaced in the target text using a preset identifier. Different genders can correspond to different preset identifiers. For example, male can correspond to "#", and female can correspond to "@". When the target text is "Zhang San (a male character name) is an ordinary person", if the target text includes the character name "Zhang San", and based on the mapping relationship, the attribute of "Zhang San" is determined to be male, then the preset identifier "#" can be used to replace "Zhang San", resulting in the replaced target text "# is an ordinary person". This assists the text annotation model in identifying the gender orientation of each target text, thereby better determining the target category of each target text.

[0121] In this embodiment of the invention, the preset category may include multiple subcategories; among the multiple subcategories, there may be subcategories corresponding to the gender of the person. For example, the preset category is a negative semantic category, which includes multiple negative semantic subcategories, one of which is a subcategory of being unfriendly to women.

[0122] Furthermore, the classification prompts can include multiple sub-classification prompts corresponding to multiple subcategories; among these multiple sub-classification prompts are sub-classification prompts corresponding to the gender of the person. For example, when the preset category is a negative semantic category, the classification prompts are the classification prompts corresponding to the negative semantic category, which can specifically include classification prompts for multiple negative semantic subcategories. When one of the multiple negative semantic subcategories is the "unfriendly to women" subcategory, one of the classification prompts for the multiple negative semantic subcategories can be the classification prompt for that "unfriendly to women" subcategory.

[0123] To enable the text annotation model to distinguish between different genders, the classification prompts can also include a pre-defined correspondence between the identifier and the person's gender; this allows the text annotation model to accurately distinguish between different genders based on the pre-defined correspondence between the identifier and the person's gender.

[0124] Therefore, after replacing the names of people in the target text, inputting the category suggestions and the target text into the text annotation model can include:

[0125] The categorized hints and the replaced target text are input into the text annotation model. The text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the person's gender based on the correspondence between each subcategory hint, preset label and the person's gender. If so, the subcategory corresponding to the person's gender is used as the target category of the target text for annotation.

[0126] When the classification prompt includes multiple subcategories, the text annotation model can predict the subcategories of the preset category to which the target text belongs. Specifically, the text annotation model can predict the subcategories of the subcategories of the preset category to which the replaced target text belongs. When the preset category includes a subcategory corresponding to a person's gender, the model can accurately determine the subcategory to which the replaced target text belongs, especially whether the replaced target text belongs to the subcategory corresponding to a person's gender, by using the preset identifiers included in the replaced target text and the correspondence between the preset identifiers and the gender of the task.

[0127] In this embodiment of the invention, replacing the names of people in the target text with preset identifiers enables the text annotation model to better learn the gender bias of the target text, thereby making a more accurate prediction of the category to which the target text belongs. Therefore, even if the preset category (multiple subcategories of the preset category) does not include a category corresponding to the gender of a person, the preset identifiers can still be used to replace the names of people in the target text, allowing the text annotation model to combine the gender bias of the target text to predict the category to which the target text belongs, thus improving prediction accuracy.

[0128] S104, in the output results, a target text with a target category of a preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples.

[0129] The output of a text annotation model can include the annotated target text. In one possible implementation, the target text can be annotated in the form of a list, meaning the target text and its category can be output as a list. In another possible implementation, the form output by the text annotation model can be in JSON (JavaScript Object Notation) format.

[0130] To assist text annotation models in determining the target category of text, a judgment criterion can be used as an intermediate layer. This layered approach helps the text annotation model fully learn the classification hints and the semantics of the target text, thereby enabling the model to more accurately determine the target category. In other words, when outputting results, the text annotation model can first provide the judgment criterion, and then output the target category. The judgment criterion can be understood as the basis for determining whether the target text belongs to the target category, obtained by the text annotation model after learning the classification hints and the semantics of the target text.

[0131] Furthermore, in the form output by the text annotation model, for each target text input to the text annotation model, the original text of the target text can be given first, then the criteria for judging whether the target text belongs to a subcategory can be given, and finally the target category to which the target text belongs can be given.

[0132] In one possible implementation, the judgment criteria identified by the text annotation model can be utilized. These criteria can include characters / words appearing in the target text. That is, after learning from the classification prompts and the target text, the text annotation model determines that the target text belongs to the target category because these characters / words are present in the target text. Furthermore, the judgment criteria of target text belonging to the preset category can be used to update the preset keywords of the preset category, thereby maintaining the timeliness of the preset keywords.

[0133] In one possible implementation, the form output by the text annotation model can be as shown in Table 1.

[0134] Table 1. Example of Text Annotation Model Output Form

[0135] He's really ugly. Ugly 1 …… …… ……

[0136] In this invention, the "raw_text" field represents the original target text; the "reason" field represents the judgment criterion; and the "pred" field represents the subcategory. In this embodiment, the subcategory can be represented by a serial number, with different serial numbers corresponding to different subcategories. This embodiment does not impose any restrictions on the fields representing the target text, the judgment criterion, or the subcategory.

[0137] For the form output by the text annotation model, each target text in the form, along with its corresponding sub-category, can be extracted as training samples. This embodiment of the invention does not limit the specific form of the text annotation model.

[0138] In order to obtain a classification model capable of accurately classifying a preset category, this embodiment of the invention selects training samples of the preset category to train the initial classification model. Therefore, based on the output of the text annotation model, target text with a target category of the preset category and the preset category can be selected as a set of training samples to obtain multiple sets of training samples belonging to the preset category.

[0139] The process of generating training samples is achieved through the above steps S101-S104.

[0140] In one possible implementation, this embodiment of the invention uses the original text as the bullet comment and a preset category as a negative semantic category. The negative semantic category includes multiple negative semantic subcategories, one of which is a female-unfriendly subcategory corresponding to the character's gender. For example, the following example, including the target text (bullet comment) and classification prompts, is input into the text annotation model:

[0141] You are a video website's content moderation expert. Your task is to categorize every bullet comment in a movie or variety show into three categories:

[0142] I. Universally Negative Values

[0143] 1. Attacks or sarcastic remarks targeting gender groups or professions. For example, "Men are prone to cheating," or "Cheating is always a woman's problem." Note that attacks on specific individuals are *acceptable*, such as "These women are bad people."

[0144] 2, [...]

[0145] 3, [...]

[0146] 4, [...]

[0147] When judging negative values, do not consider the tone; just look at whether the core content hits the four points mentioned above.

[0148] II. Unfriendly to women

[0149] 1. Attacks, discrimination, and prejudice against women, and speaking ill of female figures. For example, "This woman really brought this on herself," "She's too promiscuous," "@She's such a hypocrite, she got the benefit and is now acting innocent."

[0150] 2, [...]

[0151] 3, [...]

[0152] III. Blank categories that do not belong to the above two categories

[0153] Please determine the category of each comment, where "#" represents a male character and "@" represents a female character. Output a JSON list. For each comment, first provide the "raw_text" field representing the original sentence, then the "reason" field representing the reason for the judgment, and finally the "pred" of the category index (1, 2, 3).

[0154] {danmu}

[0155] In the example above, {danmu} represents the bullet screen content to be annotated, i.e., the target text. The other content besides {danmu} is a category hint, where "#" represents male characters and "@" represents female characters" is a preset correspondence between identifiers and character gender; [...] indicates omission. This embodiment of the invention does not limit the omitted content, the number of categories, or the specific number of items included in each category; technicians can set these according to actual needs. It can be understood that the above "universal negative values," "unfriendly to women," and "blank categories not belonging to the above two categories" are subcategories corresponding to the negative semantic categories.

[0156] Therefore, by utilizing a text annotation model combined with classification hints and target text, the annotation of negative semantic category bullet comments is automatically performed, generating training samples. This provides a high-quality negative semantic category bullet comment dataset for subsequent training of the initial classification model, solving the problem of the lack of high-quality negative bullet comment classification datasets, i.e., negative slot data (data that helps machines understand human intentions from a semantic perspective), in related technologies. In other words, this embodiment of the invention uses a text annotation model combined with classification hints to annotate the target text, obtaining high-quality classification model training samples, thus solving the problem of the lack of high-quality classification datasets (training samples) in related technologies.

[0157] After obtaining the training samples, this embodiment of the invention describes the process of training the classification model using the training samples.

[0158] S105, Select training samples and repeat the following steps: For any selected set of training samples, input the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples;

[0159] In the initial classification model, the weight matrix is ​​obtained by superimposing the general weight matrix and the increment matrix; the increment matrix corresponds to the category of multiple training samples.

[0160] The initial classification model can be obtained by superimposing an increment matrix onto a general classification model. A general classification model can be understood as one that cannot accurately classify certain specific categories, but can perform simple classifications of different categories. Therefore, if the weight matrix of the general classification model is considered as a general weight matrix, the weight matrix of the initial classification model is obtained by superimposing the general weight matrix and the increment matrix.

[0161] Before inputting training samples into the initial classification model to obtain the predicted classification results corresponding to the training samples, an increment matrix corresponding to the categories of multiple training samples can be created first. This increment matrix is ​​initially a zero matrix. Then, a general weight matrix is ​​superimposed on the increment matrix to obtain the weight matrix of the initial classification model. In this way, different categories of training samples can correspond to different increment matrices. Furthermore, by superimposing the different increment matrices obtained from the final training with the general weight matrix, different classification models can be obtained, which can be used to accurately classify different categories respectively.

[0162] For example, training samples with a pre-defined negative semantic category can correspond to an increment matrix of the negative semantic category. After the increment matrix of the negative semantic category is trained, the classification model obtained by superimposing the increment matrix of the negative semantic category with the general weight matrix can achieve accurate classification of the negative semantic category.

[0163] The increment matrix can include a first matrix and a second matrix. Specifically, the first matrix can be a zero matrix, and the second matrix can be a Gaussian matrix. The product of the first and second matrices can be used as the increment matrix. Then, multiplying the zero matrix by the Gaussian matrix yields the increment matrix initially set to zero. The increment matrix has the same dimension as the general weight matrix, while the first and second matrices have lower dimensions. This ensures that in the initial training phase, only the general weight matrix affects the prediction of the training samples.

[0164] In this embodiment of the invention, the general classification model may include the BERT classification model (Bidirectional Encoder Representations from Transformers, a self-encoding language model).

[0165] S106, calculate the loss value between the predicted classification result and the preset categories included in the training samples, update the increment matrix using the loss value, until the termination condition is met, and obtain the classification model.

[0166] Regarding the output of the S103 text annotation model, the target category of the target text can be understood as the ground truth value of the target text. Furthermore, when a training sample corresponds to a preset category, the ground truth value of the training sample is the preset category.

[0167] Typically, when retraining a pre-trained model, the weight matrix of the entire pre-trained model is updated, gradually bringing the model closer to convergence. However, updating model parameters in this way involves adjusting all parameters, making it very inflexible.

[0168] In this embodiment of the invention, since the weight matrix of the initial classification model is obtained by superimposing the general weight matrix and the newly created increment matrix, when updating the weight matrix of the initial classification model using the loss value between the predicted classification result and the preset categories included in the training samples, only the increment matrix in the weight matrix of the initial classification model needs to be updated.

[0169] Furthermore, since the dimensions of the first and second matrices constituting the increment matrix are much smaller than those of the general weight matrix, this embodiment of the invention only requires training on two low-dimensional matrices. This can be understood as meaning that this training method only requires fine-tuning a small number of parameters to achieve good training results for the initial classification model.

[0170] In this embodiment of the invention, the increment matrix can be created using the LoRA (low-rank adaptation) fine-tuning method. That is, the LoRA fine-tuning method can be used to retrain the general classification model to achieve S105-S106. When the general classification model is the BERT classification model, this embodiment of the invention uses training samples and the LoRA fine-tuning method to train the BERT classification model, obtaining a self-developed model for the category to which the training samples belong.

[0171] The termination condition can be preset. When the termination condition is met, the optimization process of the initial classification model can be considered to have ended, that is, the initial classification model tends to converge. At this point, training of the initial classification model can be stopped, and the classification model is obtained.

[0172] The termination condition is met when the cross-entropy loss function between the predicted classification result and the preset categories included in the training samples is calculated and the loss value no longer decreases. In other words, when the cross-entropy loss function no longer decreases, the initial classification model can be considered to have converged, the retraining process of the initial classification model ends, the initial classification model optimization is complete, and the classification model is obtained.

[0173] Through steps S105-S106, the classification model is trained using training samples to obtain a classification model that can accurately classify the preset categories corresponding to the training samples.

[0174] This invention utilizes a text annotation model to annotate target text within a preset category, avoiding its involvement in the extensive text classification process and saving computational resources. Training samples are determined based on the output of the text annotation model, and the initial classification model is further trained, progressively updating the increment matrix to obtain a self-developed classification model capable of accurately classifying the preset categories. When using this model for subsequent classification, lengthy classification prompts are no longer required, avoiding repetitive analysis of these prompts and thus saving computational resources needed for text classification.

[0175] In one feasible approach, acquiring multiple target texts may include: acquiring multiple target bullet comments; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, resulting in bullet comments including general negative keywords; acquiring classification hints corresponding to preset categories may include: acquiring negative semantic category classification hints; the negative semantic category classification hints include multiple negative semantic subcategories and sub-category hints for each negative semantic subcategory; inputting the classification hints and target texts into a text annotation model to obtain the output of the text annotation model may include: inputting the negative semantic category classification hints and multiple target bullet comments. Input the GPT language model to obtain the output form of the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comment based on the negative semantic category classification prompt and the target bullet comment; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment; in the output result, a target text with a target category of a preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples, which may include: in the form, selecting a target bullet comment with a target sub-category of a negative semantic sub-category, and the negative semantic sub-category of the target bullet comment as a set of training samples to obtain multiple sets of training samples.

[0176] A preset number of TV series, such as 10 series with frequently appearing negative semantic category bullet comments, can be selected in advance to maintain a general negative high-frequency word list (preset word list), which includes multiple preset general negative keywords. Using these multiple preset general negative keywords, each original bullet comment is filtered to obtain dataset A. Dataset A contains bullet comments including the preset general negative keywords, and each bullet comment in dataset A is used as the target text.

[0177] Furthermore, based on the correspondence between character names and attributes in the TV series in this embodiment of the invention, the character names appearing in the bullet comments can be replaced. Specifically, male characters can be replaced with "#" and female characters with "@", resulting in dataset B.

[0178] For the negative semantic category indicated by dataset A (dataset B), negative semantic category classification hints can be obtained; negative semantic category classification hints include multiple negative semantic subcategories and sub-classification hints for each negative semantic subcategory.

[0179] For example, write a classification prompt_L for labeling bullet comments, and then input prompt_L and dataset B into a text annotation model to label each target text in dataset B.

[0180] The text annotation model can be the GPT language model, as described in this embodiment of the invention. The target bullet comment and the classification hints for the negative semantic category can be input into the GPT language model to obtain a form output by the GPT language model. This form includes the annotated target bullet comment; specifically, it can include the target bullet comment and its target category (target subcategory). The annotated target bullet comment can be understood as the annotated dataset B.

[0181] Furthermore, in this embodiment of the invention, an initial classification model, such as the BERT classification model, can be trained using the labeled dataset B. The initial classification model can then be fine-tuned using the LoRA fine-tuning method to obtain a self-developed classification model for classifying bullet comments in the negative semantic category, specifically, the negative semantic category includes multiple negative semantic subcategories.

[0182] The classification model training method provided in this invention uses the GPT language model to automatically annotate target text and generate training samples, thus solving the problem of missing high-quality training samples. It uses an initial classification model combined with the LoRA fine-tuning method for targeted fine-tuning. The training process only requires adjusting a small number of parameters to obtain a classification model with good classification performance, effectively reducing the cost of model training.

[0183] In this embodiment of the invention, taking the original text as the bullet screen and the preset category as the negative semantic category as an example, and combining the classification prompt examples involved in S101-S104, the classification performance of the classification model trained by the closed-source large language model (such as the GPT language model) and the classification model of this embodiment of the invention in related technologies is tested. The test results are shown in Tables 2 and 3.

[0184] Table 2. Test Results of Closed-Source Large Language Model

[0185] negative values 0.51 0.77 0.61 Unfriendly to women 0.79 0.67 0.72 Normal bullet comments 0.77 0.68 0.72 average 0.69 0.7 0.68

[0186] Table 3. Classification Model Test Results

[0187] negative values 0.78 0.77 0.77 Unfriendly to women 0.73 0.85 0.78 Normal bullet comments 0.84 0.77 0.8 average 0.78 0.8 0.79

[0188] Among them, accuracy rate represents the model's classification result for the subcategory to which the bullet comment belongs, and can characterize the accurate proportion of the subcategory to which the bullet comment actually belongs.

[0189] When bullet comments are classified as having negative semantic categories, displaying these opinions would negatively impact the viewing experience for normal viewers. Therefore, displaying these bullet comments can be avoided; technical personnel can recall them. Thus, recall rate represents the proportion of recalled bullet comments out of all bullet comments classified as having negative semantic categories. A higher recall rate generally indicates a better classification performance of the model.

[0190] f1 represents the average precision and recall for each subclass; average represents the average precision, recall, and f1 for each subclass.

[0191] It can be seen that, compared with related technologies, the classification model trained in this embodiment of the invention has higher accuracy in classifying subcategories, that is, a higher accuracy in judging whether a bullet comment belongs to a subcategory of negative semantic category. Consequently, among the bullet comments of the total negative semantic category, the number of bullet comments that do not actually belong to the negative semantic category is smaller. Therefore, when recalling bullet comments that actually belong to the negative semantic category, the proportion of recalled bullet comments is higher, that is, the recall rate is also higher. It can be seen that when using the classification model trained in this embodiment of the invention for classification, the classification metrics (precision and recall rate) both exceed the classification method of closed-source large language model + prompt, and the classification effect is better.

[0192] When classifying texts with negative semantic categories, such as "Why do you say women are all spendthrifts?", classification models are prone to misclassification. That is, the text to be classified actually points to a non-negative semantic category, but the classification model identifies it as a negative semantic category.

[0193] Furthermore, to improve the classification accuracy of the classification model, the training sample set can be expanded. Specialized training samples can be constructed for some common bad cases (results that differ from expectations during the inference stage). These specialized training samples can then be used to further optimize the classification model, thereby improving its classification accuracy.

[0194] like Figure 2 As shown, this embodiment of the invention provides a classification method, which may include:

[0195] S201, Obtain the text to be classified.

[0196] S202, Input the text to be classified into the classification model, use the classification model to classify the text to be classified, and obtain the classification result of each text to be classified;

[0197] The classification model was obtained using the classification model training method described in S101-S106 above.

[0198] The classification method provided in this invention only requires inputting the text to be classified into a self-developed classification model, and the classification model can be used to classify the text without having to input a long classification prompt at the same time, thereby saving the computing resources required for text classification.

[0199] Reference Figure 2 The present invention provides a detailed description of the classification method in this embodiment.

[0200] S201, Obtain the text to be classified.

[0201] The text to be categorized can include text from comment sets and bullet screen comment sets. For example, for a newly released TV series, the bullet screen comments of that series can be used as the text to be categorized.

[0202] S202, Input the text to be classified into the classification model, use the classification model to classify the text to be classified, and obtain the classification result of each text to be classified;

[0203] The classification model was obtained using the classification model training method described in S101-S106 above.

[0204] Based on the foregoing analysis, different categories can correspond to different increment matrices, resulting in different classification models used to accurately classify different categories. Therefore, before classifying text, one can select an increment matrix that meets the classification requirements, i.e., a classification model that meets the requirements, to classify the text.

[0205] In this embodiment of the invention, the text to be classified may include bullet comments to be classified, and the selected classification model may include a classification model for classifying bullet comments trained based on multiple labeled bullet comments. Specifically, this classification model can be used to achieve accurate classification of negative semantic categories.

[0206] After inputting the text to be classified, i.e. the bullet comments to be classified, into the classification model, the classification results for each bullet comment can be obtained. For bullet comments whose classification results indicate a negative semantic category, it is generally believed that displaying these bullet comments will reduce the user's viewing experience. Therefore, the display priority of bullet comments whose classification results indicate a negative semantic category can be reduced. For example, the display priority of these bullet comments can be set to the lowest, and they can not be displayed.

[0207] In existing technologies, text classification utilizes classification prompts and closed-source large language models, such as the GPT-4 language model. However, the output of the closed-source large language model is merely the classification result it determines based on the input classification prompts, which best satisfies the prompts. For some text to be classified, although the model identifies it as belonging to a certain category, this determination may only be relative to the determinations of other categories. Simply put, the text to be classified may not perfectly match any of the categories included in the classification prompts.

[0208] However, after obtaining the classification result of the text to be classified, the device cannot judge the degree of matching between the text to be classified and the classification result based solely on the classification result. Consequently, in some scenarios where the display priority of a certain category needs to be reduced, even if the text to be classified does not match the category very well, its display priority will still be reduced, resulting in misjudgment or omission.

[0209] Therefore, in this embodiment of the invention, for unclassified bullet comments whose classification results indicate negative semantics, the confidence level of each unclassified bullet comment belonging to the classification result can be calculated. The higher the confidence level, the more reliable the classification result of the bullet comment. Therefore, it can be considered that displaying unclassified bullet comments with high confidence levels will affect the user's viewing experience. Thus, the display priority of unclassified bullet comments with confidence levels higher than a preset confidence threshold can be reduced to avoid displaying these bullet comments. The preset confidence threshold can be set by technical personnel according to actual needs to simultaneously consider the quality of displayed bullet comments and the number of bullet comments to be avoided. This embodiment of the invention does not impose any limitations on this.

[0210] The classification method provided in this invention can determine whether to distribute a bullet comment based on the bullet comment classification result and confidence threshold, thereby avoiding the display of bullet comments with negative semantic categories, improving the bullet comment atmosphere, increasing the bullet comment opening rate and reducing the number of bullet comment reports.

[0211] In one possible implementation, the trained classification model can also include a classification model for positive semantic categories. For text to be classified whose classification results indicate a positive semantic category, its display priority can be increased. For example, when there are many texts to be classified, those indicating a positive semantic category can be displayed first; display effects can be added to highlight these texts, etc., thereby creating a better community atmosphere.

[0212] like Figure 3 As shown, this invention provides a classification model training device, which may include:

[0213] The first acquisition module 301 is used to acquire multiple target texts; the multiple target texts include preset keywords belonging to preset categories;

[0214] The second acquisition module 302 is used to acquire category prompts corresponding to preset categories;

[0215] The annotation module 303 is used to input the classification hints and target text into the text annotation model and obtain the output results of the text annotation model; wherein, the text annotation model predicts and annotates the target category of each target text based on the input classification hints and target text;

[0216] The third acquisition module 304 is used to take a target text with a target category of a preset category and the preset category as a set of training samples in the output result to obtain multiple sets of training samples;

[0217] The prediction module 305 is used to select training samples and repeatedly perform the following steps: for any selected set of training samples, input the training samples into the initial classification model to obtain the prediction classification result corresponding to the training samples; wherein, the weight matrix in the initial classification model is obtained by superimposing a general weight matrix and an increment matrix; the increment matrix corresponds to the categories of multiple training samples.

[0218] The update module 306 is used to calculate the loss value between the predicted classification result and the preset categories included in the training samples, and to update the increment matrix using the loss value until the termination condition is met, thus obtaining the classification model.

[0219] Optionally, the first acquisition module 301 is specifically used to acquire multiple original texts; filter the multiple original texts using preset keywords, and retain the original texts including the preset keywords as target texts; the preset keywords are keywords of preset categories.

[0220] Optionally, the text annotation model is specifically used to segment the classification prompts and target text into words to obtain a word segmentation set; using the word vectors corresponding to each word segmentation in the word segmentation set, a target matrix corresponding to the classification prompts and target text is generated; the confidence level of the target text belonging to each category is predicted based on the target matrix; the category with the highest confidence level is taken as the target category of the target text, and the target category is labeled accordingly for the target text.

[0221] Optionally, the device further includes: a replacement module, used to identify the names of people in each target text before inputting the classification prompts and target text into the text annotation model; query the gender of the person indicated by the name of the person; and replace the name of the person in the target text with a preset identifier corresponding to the gender of the person.

[0222] The preset categories include multiple subcategories; among these subcategories are subcategories corresponding to the character's gender; the classification hints include multiple subcategories corresponding to each of the multiple subcategories; among these subcategories are subcategories corresponding to the character's gender; the classification hints also include the correspondence between preset identifiers and the character's gender.

[0223] The annotation module 303 is specifically used to input the classification prompts and the replaced target text into the text annotation model; the text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the gender of the person based on the correspondence between each subcategory prompt, the preset identifier and the gender of the person. If so, the subcategory corresponding to the gender of the person is used as the target category of the target text for annotation.

[0224] Optionally, the device further includes:

[0225] A module is created to generate an increment matrix corresponding to the preset categories of multiple training samples before inputting training samples into the initial classification model and obtaining the predicted classification results corresponding to the training samples. The increment matrix is ​​initially a zero matrix. A general weight matrix is ​​superimposed on the increment matrix to obtain the weight matrix of the initial classification model.

[0226] Optionally,

[0227] The first acquisition module 301 is specifically used to acquire multiple target bullet comments; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, and then including general negative keywords.

[0228] The second acquisition module 302 is specifically used to acquire negative semantic category classification hints; the negative semantic category classification hints include multiple negative semantic subcategories and sub-classification hints for each negative semantic subcategory;

[0229] The annotation module 303 is specifically used to input negative semantic category classification prompts and multiple target bullet comments into the GPT language model to obtain a form output by the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comments based on the negative semantic category classification prompts and target bullet comments; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment;

[0230] The third acquisition module 304 is specifically used to select a target barrage whose target subcategory is a negative semantic subcategory from the form, and the negative semantic subcategory of the target barrage as a set of training samples to obtain multiple sets of training samples.

[0231] The classification model training device provided in this invention only uses a text annotation model to annotate target text of a preset category, avoiding its participation in the classification process of a large number of texts, thus saving computational resources. Training samples are determined based on the output of the text annotation model, and the initial classification model is further trained. The incremental matrix in the initial classification model is gradually updated to obtain a self-developed classification model capable of accurately classifying the preset categories. When using this classification model for subsequent classification, there is no need to input lengthy classification prompts to classify the input text, avoiding the model repeatedly analyzing classification prompts, thereby saving computational resources required for text classification.

[0232] This invention also provides an electronic device, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0233] Memory 403 is used to store computer programs;

[0234] The processor 401 is used to execute the program stored in the memory 403 to implement the above-mentioned classification model training method.

[0235] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0236] The communication interface is used for communication between the aforementioned terminal and other devices.

[0237] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0238] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0239] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the classification model training methods described in the above embodiments.

[0240] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the classification model training methods described in the above embodiments.

[0241] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0242] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0243] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0244] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A classification model training method, characterized in that, The method includes: Acquire multiple target texts; the multiple target texts include preset keywords belonging to preset categories; Obtain the category suggestion corresponding to the preset category; The classification hints and the target text are input into the text annotation model to obtain the output of the text annotation model; wherein, the text annotation model predicts and labels the target category of each target text based on the input classification hints and the target text; In the output results, a target text whose target category is the preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples; The following steps are repeated with the selected training samples: for any selected set of training samples, the training samples are input into the initial classification model to obtain the predicted classification result corresponding to the training samples; wherein, the weight matrix in the initial classification model is obtained by superimposing a general weight matrix and an increment matrix; the increment matrix corresponds to the category of the multiple sets of training samples; Calculate the loss value between the predicted classification result and the preset categories included in the training samples, update the increment matrix using the loss value, and continue until the termination condition is met to obtain the classification model; The acquisition of multiple target texts includes: Multiple target bullet comments are obtained; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, and the bullet comments include general negative keywords. The step of obtaining the category hints corresponding to the preset category includes: Obtain negative semantic category classification hints; the negative semantic category classification hints include multiple negative semantic subcategories and sub-classification hints for each negative semantic subcategory; The step of inputting the classification prompts and the target text into the text annotation model to obtain the output of the text annotation model includes: The negative semantic category classification prompt and the multiple target bullet comments are input into the GPT language model to obtain a form output by the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comments based on the negative semantic category classification prompt and the target bullet comments; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment. In the output result, a target text whose target category is the preset category and the preset category are used as a set of training samples to obtain multiple sets of training samples, including: In the form, a target barrage whose target subcategory is the negative semantic subcategory is selected, and the negative semantic subcategory of the target barrage is used as a set of training samples to obtain multiple sets of training samples.

2. The method according to claim 1, characterized in that, The acquisition of multiple target texts includes: Retrieve multiple raw texts; Using preset keywords, the multiple original texts are filtered, and the original texts containing the preset keywords are retained as target texts; the preset keywords are keywords of preset categories.

3. The method according to claim 1, characterized in that, The text annotation model predicts and annotates the target category of each target text based on the input classification prompts and the target text, including: The classification prompts and the target text are segmented into words to obtain a segmentation set; Using the word vectors corresponding to each word in the word segmentation set, a target matrix corresponding to the classification prompt and the target text is generated; Predict the confidence level of the target text belonging to each category based on the target matrix; The category with the highest confidence level is taken as the target category of the target text, and the target category is labeled accordingly.

4. The method according to claim 1, characterized in that, Before inputting the classification prompts and the target text into the text annotation model, the method further includes: For each target text, identify the names of people in the target text; Query the gender of the person indicated by the name, and replace the name of the person in the target text with a preset identifier corresponding to the gender; The preset category includes multiple subcategories; the multiple subcategories include a subcategory corresponding to the character's gender; the classification prompts include multiple sub-classification prompts corresponding to each of the multiple subcategories; the multiple sub-classification prompts include a sub-classification prompt for the subcategory corresponding to the character's gender; the classification prompts also include the correspondence between the preset identifier and the character's gender; the step of inputting the classification prompts and the target text into the text annotation model includes: The classification prompts and the replaced target text are input into the text annotation model; the text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the person's gender based on the correspondence between each subcategory prompt, the preset identifier and the person's gender. If so, the subcategory corresponding to the person's gender is used as the target category of the target text for annotation.

5. The method according to claim 1, characterized in that, Before inputting the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples, the method further includes: Create an increment matrix corresponding to the preset categories to which multiple sets of training samples belong; the increment matrix is ​​initially a zero matrix. The weight matrix of the initial classification model is obtained by superimposing a general weight matrix with the incremental matrix.

6. A classification model training device, characterized in that, The device includes: The first acquisition module is used to acquire multiple target texts; the multiple target texts include preset keywords belonging to preset categories; The second acquisition module is used to acquire classification prompts corresponding to the preset category; The annotation module is used to input the classification prompts and the target text into the text annotation model and obtain the output results of the text annotation model; wherein, the text annotation model predicts and annotates the target category of each target text based on the input classification prompts and the target text; The third acquisition module is used to take a target text whose target category is the preset category and the preset category as a set of training samples in the output result to obtain multiple sets of training samples. The training module is used to select the training samples and repeatedly perform the following steps: for any selected set of training samples, input the training samples into the initial classification model to obtain the predicted classification result corresponding to the training samples; wherein, the weight matrix in the initial classification model is obtained by superimposing a general weight matrix and an increment matrix; the increment matrix corresponds to the category of the multiple sets of training samples; The update module is used to calculate the loss value between the predicted classification result and the preset categories included in the training samples, and update the increment matrix using the loss value until the termination condition is met to obtain the classification model. The first acquisition module is specifically used to acquire multiple target bullet comments; the target bullet comments are obtained by filtering multiple original bullet comments using preset general negative keywords, and the bullet comments include general negative keywords. The second acquisition module is specifically used to acquire negative semantic category classification prompts; the negative semantic category classification prompts include multiple negative semantic subcategories and sub-classification prompts for each negative semantic subcategory; The annotation module is specifically used to input the negative semantic category classification prompts and the multiple target bullet comments into the GPT language model to obtain a form output by the GPT language model; the GPT language model is used to predict the target sub-category of the target bullet comments based on the negative semantic category classification prompts and the target bullet comments; the form includes multiple target bullet comments and the target sub-category corresponding to each target bullet comment. The third acquisition module is specifically used to select a target barrage whose target subcategory is the negative semantic subcategory from the form, and the negative semantic subcategory of the target barrage as a set of training samples to obtain multiple sets of training samples.

7. The apparatus according to claim 6, characterized in that, The device further includes: The replacement module is used to identify the person's name in each target text before inputting the classification prompt and the target text into the text annotation model; query the gender of the person indicated by the person's name; and replace the person's name in the target text with a preset identifier corresponding to the person's gender. The preset category includes multiple subcategories; among the multiple subcategories is a subcategory corresponding to the character's gender; the classification prompts include multiple sub-classification prompts corresponding to each of the multiple subcategories; among the multiple sub-classification prompts is a sub-classification prompt for the subcategory corresponding to the character's gender; the classification prompts also include the correspondence between the preset identifier and the character's gender; The annotation module is specifically used to input the classification prompts and the replaced target text into the text annotation model; the text annotation model is used to determine whether each target text belongs to the subcategory corresponding to the person's gender based on the correspondence between each subcategory prompt, the preset identifier and the person's gender, and if so, to annotate the subcategory corresponding to the person's gender as the target category of the target text.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-5.

Images