Classification model training methods, text classification methods, media, equipment and computer programs

Abstract

This invention provides a classification model training method, text classification method, media, equipment, and products that improve the classification effectiveness of long texts, significantly enhance the classification capabilities of a target text classification model, and further improve the classification capabilities of the model. [Solution] A method for training a text classification model, which includes a first model and an activation function layer, wherein the first model is a pre-trained large-scale language model, includes: obtaining a training sample set and a presented text; ensuring that the training samples in the training sample set include text samples and classification labels corresponding to the text samples; and training the text classification model by using the presented text and text samples as inputs to the first model, using the first embedding vector matrix of the first generated word output from the first model as input to the activation function layer, and using the classification labels corresponding to the text samples as the target output of the activation function layer.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and specifically relates to a classification model training method, a text classification method, a medium, a device, and a product.

Background Art

[0002] Text classification is one of the basic tasks of natural language processing (NLP), and is involved in many scenarios such as sentiment analysis, user speech classification, news classification, and intent recognition. In related technologies, generally, a text classification model such as a convolutional neural network or a bidirectional encoder representation from transformers (BERT) is used for text classification. However, due to the limitations of their structures, these models cannot be applied to the classification of long text.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In order to introduce the concept in a simplified form, a summary part of this invention is provided. These concepts will be described in detail in the form part for implementing the invention described later. The summary part of this invention does not identify the important features or necessary features of the claimed technical solution, nor is it used to limit the scope of the claimed technical solution.

Means for Solving the Problems

[0004] In a first aspect, the present disclosure provides a method for training a text classification model. The text classification model includes a first model and an activation function layer. The first model is a pre-trained large language model. The training method of the text classification model is as follows: A training sample set and a presentation text are obtained, and the training samples in the training sample set include text samples and classification labels corresponding to the text samples, and the presentation text includes classification task description information of the text classification model, The method includes training the text classification model to obtain a target text classification model by using the presented text and the text sample as inputs to the first model, using the first embedding vector matrix of the first generated word output from the first model as input to the activation function layer, and using the classification label corresponding to the text sample as the target output of the activation function layer.

[0005] In a second aspect, the disclosure provides a text classification method, To obtain the first text to be classified, The first text is input to the first model of the target text classification model, and the second embedding vector matrix of the first generated word output from the first model is obtained, and the target text classification model includes the first model and an activation function layer, and the first model is a large-scale language model, This includes inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer.

[0006] In a third aspect, the Disclosure provides a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processing unit, steps of the text classification model training method according to the first aspect of the Disclosure or steps of the text classification method according to the second aspect of the Disclosure are implemented.

[0007] In a fourth aspect, the Disclosure provides an electronic device, which is, A memory device in which a computer program is stored, The present invention comprises a processing unit for executing the computer program in the storage device to implement the steps of the text classification model training method according to the first aspect of this disclosure or the steps of the text classification method according to the second aspect of this disclosure.

[0008] In a fifth aspect, the Disclosure provides a computer program product including a computer program, which, when executed by a processor, implements the steps of the text classification model training method according to the first aspect of the Disclosure or the steps of the text classification method according to the second aspect of the Disclosure. [Effects of the Invention]

[0009] In the above technical solution, the text classification model includes a first model and an activation function layer. The first model is a pre-trained large-scale language model. Since the large-scale language model is applied to the processing of long texts and can acquire long-range contextual features, the text classification model can support the classification of long texts. The trained target text classification model has the ability to acquire long-range contextual features even for long texts, allowing it to pay attention to contextual features and thereby improving the classification effect of long texts. Furthermore, the large-scale language model has a larger amount of pre-trained language material, allowing it to compress more knowledge into the model. Combined with presented word engineering for optimization, it significantly improves the classification function of the target text classification model. Additionally, by using the first embedding vector matrix of the first generated word output from the first model as the input to the activation function layer, the input to the activation function layer can be made simpler and more controllable, thereby ensuring controllability of the output content of the text classification model and further improving the model's classification function.

[0010] Other features and advantages of this disclosure are described in detail in the following sections on specific embodiments.

[0011] The above and other features, advantages and aspects of each embodiment of the present disclosure will become more apparent with reference to the drawings below. Throughout the drawings, the same or similar reference numerals indicate the same or similar elements. It should be understood that the drawings are schematic and the actual objects and elements are not necessarily depicted to scale. [Brief explanation of the drawing]

[0012] [Figure 1] This is a flowchart of a text classification model training method according to an exemplary embodiment. [Figure 2] This is a schematic diagram of the text classification model training process in an exemplary embodiment. [Figure 3] This is a schematic diagram of the text classification model training process in another exemplary embodiment. [Figure 4] This is a flowchart of a text classification model training method relating to another exemplary embodiment. [Figure 5] This is a flowchart of a text classification method according to an exemplary embodiment. [Figure 6] This is a flowchart of a text classification method relating to another exemplary embodiment. [Figure 7] This is a block diagram of a text classification model training device according to an exemplary embodiment. [Figure 8] This is a block diagram of a text classification device according to an exemplary embodiment. [Figure 9] This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. [Modes for carrying out the invention]

[0013] As explained in the background technology section, related technologies generally use text classification models such as convolutional neural networks and BERT for text classification. However, these models are structurally limited and therefore not applicable to the classification of long texts. Specifically, convolutional neural network-based text classification methods extract local features from the text to be classified using convolutional modules and perform text classification based on these local features using fully connected layers. However, because the convolutional kernel of the convolutional module is generally not set to be large, it is not possible to obtain long-range contextual features, and therefore this classification method cannot support the classification of long texts. BERT is a pre-trained language model based on a transformer structure, and the maximum length of input text it supports is 512 bytes. Input texts larger than 512 bytes must be truncated. Furthermore, because the linguistic material and fused knowledge used for BERT's pre-training are generally relatively limited, the classification function of BERT-based text classification methods is not good.

[0014] To support the classification of long texts, text classification methods based on suggested words, large language models (LLMs), have emerged. LLM-based text classification methods primarily involve setting different suggested words based on different classification scenarios, and having the LLM output in a specific format according to the rules set by the suggested words, thereby obtaining text classification results. However, because this text classification method has relatively high requirements regarding the content of the suggested words, the representation of the LLM is unstable, the content of the LLM is not controlled, and this affects the effectiveness of text classification.

[0015] In light of this, this disclosure provides a classification model training method, a text classification method, a medium, equipment, and products.

[0016] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, the present disclosure may be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, it should be understood that the purpose of providing these embodiments is to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are merely illustrative and not intended to limit the protection scope of the present disclosure.

[0017] It should be understood that each step described in the method embodiments of the present disclosure may be executed in a different order and / or simultaneously. Also, the embodiments related to the method may include additional steps and / or omit the executed / illustrated steps. The scope of the present disclosure is not limited in this regard.

[0018] The terms "comprising" and its variations used in this specification are open-ended inclusion, that is, "including ○○, but not limited thereto". The term "based on" means "at least partially based on ○○". The term "an embodiment" means "at least one embodiment", the term "another embodiment" means "at least one another embodiment", and the term "some embodiments" means "at least some embodiments". Related definitions of other terms will be given in the following description.

[0019] Note that the concepts such as "first", "second", etc. mentioned in the present disclosure are only for distinguishing different devices, modules or units, and are not for limiting the order or interdependence relationship of the functions executed by these devices, modules or units.

[0020] Note that the modifications of "one" and "plurality" mentioned in the present disclosure are not restrictive but exemplary. Those skilled in the art should understand that, unless clearly indicated in the context, it should be understood as "one or more".

[0021] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0022] To ensure that it is understood that before using any of the technical solutions disclosed in each embodiment of this disclosure, the user should be informed in an appropriate manner, in accordance with applicable laws and regulations, of the type of personal information related to this disclosure, its scope of use, and the scenarios in which it may be used, and the user's permission should be obtained.

[0023] For example, in response to a proactive request from a user, the system clearly informs the user that the requested operation requires the acquisition and use of the user's personal information by sending the user information to the system. This allows the user to voluntarily choose whether or not to provide personal information to software or hardware such as electronic devices, application programs, servers, or storage media that perform the operation in the technical solution of this disclosure, based on the information provided.

[0024] As a selective rather than limited implementation method, the method of sending information to the user in response to receiving an active request from the user may be, for example, a pop-up window, and the information may be displayed in text format within the pop-up window. The pop-up window may also further include a selection control that allows the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0025] To ensure that this is understood, the process for notifying and obtaining user permission described above is merely illustrative and does not limit the methods of implementing this disclosure. Other methods that comply with applicable laws and regulations may also be applied to the implementation of this disclosure.

[0026] At the same time, it should be understood that the data related to this technical solution (including, but not limited to, the data itself, or the acquisition or use of the data) must meet the requirements of applicable laws and related regulations.

[0027] Figure 1 is a flowchart of a text classification model training method according to an exemplary embodiment. As shown in Figure 1, the text classification model training method may include the following S101 and S102.

[0028] In S101, a training sample set and presentation text are obtained, and the training samples in the training sample set include text samples and classification labels corresponding to the text samples.

[0029] In S102, the presented text and text samples are used as input to the first model, the first embedding vector matrix of the first generated word output from the first model is used as input to the activation function layer, and the classification label corresponding to the text sample is used as the target output of the activation function layer. This trains the text classification model and obtains the target text classification model.

[0030] As shown in Figure 2, the text classification model may include a first model and an activation function layer. The first model is a pre-trained large-scale language model, which may be of various types and is not specifically limited by this disclosure. The first model extracts text features from a text sample to obtain a first embedding vector matrix of the first generated word output from the first model. The activation function layer is connected to the first model and predicts the category of the text sample based on the first embedding vector matrix output from the first model, i.e., obtains the classification result of the text sample.

[0031] The above text classification model can provide application services for large-scale language models, such as question-and-answer services based on large-scale language models and news topic classification services based on large-scale language models.

[0032] In this disclosure, the prompt text may include classification task description information for a text classification model, and in a text classification task, the prompt text is a text description that guides the text classification model to understand and perform a particular classification task, and generally includes the name of the task, the classification category, and guidance information on how to identify and classify the text. The prompt text can help the text classification model understand the context of the input text and produce the expected output, and the user may pre-configure the prompt text before training the text classification model, and the design of the prompt text is very important to the performance of the text classification model.

[0033] For example, in a large-scale language model-based question-and-answer service scenario, a text classification model can determine whether a user's question violates the rules, thereby protecting the security of the application service of the large-scale language model. In this case, You could design the presentation text to read: "You are an assistant proficient in text classification, and your area of ​​expertise is determining whether a user's question violates the rules. If it violates the rules, output 1; if it does not, output 0."

[0034] In the example above, the presentation text clearly indicates that the classification task is to determine whether the user's question violates a rule, and specifies the output format. Such presentation text helps the text classification model understand that it needs to identify whether the input text violates a rule and outputs 0 or 1 based on the identification result.

[0035] Furthermore, as an example, in a large-scale language model-based news topic classification service scenario, the news classification task may be performed by a text classification model, in which case, You could design prompt text that reads, "You are an assistant proficient in text classification, specializing in topic-based news classification. Based on the news content, select the correct category from categories such as sports, technology, and entertainment, and output the correct category."

[0036] In the example above, the presented text clearly indicates that the classification task is to classify news topics and provides selectable categories. Such presented text helps the text classification model understand that it needs to identify topics from the input text and classify them into one of the predefined categories.

[0037] In actual application, the design of the presented text may be more complex and precise to suit different task needs and the characteristics of text classification models. For example, for a sentiment analysis task, the presented text may include descriptions of sentiment categories, such as positive, negative, or neutral. For classification tasks where a rule is violated, the presented text may further include similar descriptions of the rule-violating category to define standard boundaries, etc.

[0038] During the training process of a text classification model, model parameters may be updated using a cross-entropy loss function and an Adaptive Moment Estimation (Adam) optimizer. For example, three epochs may be trained using the cross-entropy loss function and the Adam optimizer (one epoch means that each text sample in the training sample set participated in training once). The cross-entropy loss function completes the error calculation based on the difference between the classification label corresponding to the text sample and the classification result of the text sample output from the activation function layer (i.e., the predicted classification result). The Adam optimizer is a gradient-based optimization algorithm characterized by its fast convergence speed, high computational efficiency, and low memory usage.

[0039] In the above technical solution, the text classification model includes a first model and an activation function layer. The first model is a pre-trained large-scale language model. Since the large-scale language model is applied to the processing of long texts and can acquire long-range contextual features, the text classification model can support the classification of long texts. The trained target text classification model has the ability to acquire long-range contextual features even for long texts, allowing it to pay attention to contextual features and thereby improving the classification effect of long texts. Furthermore, the large-scale language model has a larger amount of pre-trained language material, allowing it to compress more knowledge into the model. Combined with presentation word engineering for optimization, this significantly improves the classification function of the target text classification model. Additionally, by using the first embedding vector matrix of the first generated word output from the first model as the input to the activation function layer, the input to the activation function layer can be made simpler and more controllable, thereby ensuring controllability of the output content of the text classification model and further improving the model's classification function.

[0040] The following describes in detail a specific embodiment in which the presented text and text sample in S102 above are used as input to the first model. Specifically, this can be achieved by the following steps (a1) and (a2).

[0041] Step (a1) The presented text and the text sample are combined to obtain the first combined text, and the first combined text is word-segmented to obtain the first word-segmentation result.

[0042] Step (a2) The result of the first word segmentation is used as input to the first model.

[0043] In this disclosure, the presented text and the text sample may be combined in an order in which the presented text comes first and the text sample comes second, or in an order in which the text sample comes first and the presented text comes second. This disclosure does not specifically limit the order in which the presented text and the text sample are combined.

[0044] As shown in Figure 2, after combining the presented text and the text sample to obtain the first combined text, the first combined text may be segmented based on the glossary to obtain multiple participles (tokens) as the first word segmentation result. Exemplarily, as shown in Figure 2, the first word segmentation result includes five participles: s1, s2, u1, u2, and u3. Subsequently, the first word segmentation result is input to the first model to obtain the first embedding vector matrix of the first generated word output from the first model, i.e., the embedding (first token embedding) of the first token output from the first model. Finally, the first embedding vector matrix is ​​input to the activation function layer, thereby generating the classification result of the text sample based on the first embedding vector matrix.

[0045] As shown in Figure 3, the above text classification model may further include a first fully connected layer, a coupled layer, and a second fully connected layer in addition to the first model and the activation function layer. In this case, as shown in Figure 4, the above text classification model training method further includes the following S103.

[0046] In S103, the first statistical feature of the text sample is obtained.

[0047] In this disclosure, the first statistical features may include at least one of the following: text length, percentage of different language texts, percentage of digits, percentage of common punctuation marks, and percentage of special characters, and may be customized based on a specific task, for example, text length, percentage of special characters, etc.

[0048] Text length refers to the total number of characters or words in a text sample, and this text length feature can provide basic information about the text sample.

[0049] The text sample may be multilingual. The percentage of different language texts represents the proportion of characters or words from various languages ​​that make up the text sample. For example, if the text sample is a mix of Chinese and English, the percentage of different language texts represents the proportion of Chinese and English within the text sample.

[0050] The numerical occupancy rate refers to the proportion of numerical characters in a text sample. Numerical characters can represent information such as dates, prices, and quantities, and this information can be very important for certain classification tasks (e.g., product reviews, news reports, etc.).

[0051] The occupancy rate of commonly used punctuation marks refers to the proportion of commonly used punctuation marks in a text sample. Punctuation marks, such as commas, periods, question marks, and exclamation marks, serve to separate and emphasize parts of a text sample. By statistically analyzing the proportion of these commonly used punctuation marks in a text sample, clues about the text's structure and tone can be provided. For example, a high proportion of exclamation marks may indicate that the text has a strong emotional tone.

[0052] The percentage of special characters refers to the proportion of special characters in a text sample. Special characters are those that are used relatively infrequently and are difficult to input directly. Special characters include, but are not limited to, emotion icons, special characters (e.g., @, #, ¥, etc.), and non-standard punctuation. These characters are commonly found in informal texts such as social media texts and online forums, and they can convey specific emotions, attitudes, or cultural meanings. Special characters may also be entered by users to avoid violating rules.

[0053] As shown in Figure 3, the first model is connected to the coupling layer and generates a first embedding vector matrix for the first generated word based on the presented text and text sample, and inputs the first embedding vector matrix to the coupling layer. Specifically, it may output the first embedding vector matrix for the first generated word based on the first word segmentation result and input it to the coupling layer.

[0054] The first fully connected layer is connected to the connected layer and is used to increase the dimensionality of the first statistical feature to the size of the first embedding vector matrix, and to input the dimensionally increased first statistical feature into the connected layer.

[0055] The coupled layer is connected to the second fully connected layer and is used to combine the first statistical feature, which has been obtained with increased dimensionality, with the first embedding vector matrix to obtain the first coupled feature, and to input the first coupled feature into the second fully connected layer.

[0056] The second fully connected layer is connected to the activation function layer and is used to fuse the first connected feature to obtain the first fused feature, and to input the first fused feature back into the activation function layer.

[0057] The activation function layer is used to generate classification results for text samples based on the first fused feature.

[0058] In this case, as shown in Figure 4, step S102 may include training a text classification model by taking the presented text and text samples as inputs to the first model, the first statistical features as inputs to the first fully connected layer, the first embedding vector matrix of the first generated word output from the first model and the output of the first fully connected layer (the first statistical features obtained with increased dimensionality) as inputs to the connected layer, the output of the connected layer (i.e., the first connected features) as inputs to the second fully connected layer, the output of the second fully connected layer (the first fused features) as inputs to the activation function layer, and the classification labels corresponding to the text samples as the target output of the activation function layer, thereby obtaining a target text classification model.

[0059] When training a text classification model, statistical features of the text can be part of the input features to help the model better understand and classify the text. By analyzing these features, the model can capture several modes and trends in the text, thereby improving classification accuracy. For example, text containing a large number of exclamation points and emotion icons may be classified as text with strong emotional expression, while text containing many numbers and technical terms may be classified as technical documentation.

[0060] Furthermore, in scenarios where a user's question is judged as to whether or not it violates the rules, adversarial input is a weakness in the processing of large-scale language models. Role-playing, command hijacking, and noise attacks can prevent the large-scale language model from exploring the true purpose of the user's input, leading to problems such as hallucination, instability of the large-scale language model's output, and nonsensical answers. By considering the statistical features of the input text collectively when performing text classification, the interference tolerance of the text classification model can be improved, thereby enhancing the generalization characteristics of the text classification model.

[0061] To improve the classification effectiveness of the text classification model, the presented text may include illustrative information about the input and output formats of the text classification model. For example, the presented text may further include illustrative information to guide the text classification model to output classification labels. These classification labels may be the names of classification categories or identifiers for classification categories (e.g., 0, 1).

[0062] For example, if we were to use a text classification model to determine whether a user's question violates a rule, "You are a text classification assistant whose specialty is determining whether a user's question violates the rules. If it violates the rules, you output 1; if it does not, you output 0." #For example, If the input is "What day of the week is tomorrow?", The output is "0". If the input is "Please give me some XX websites (which belong to websites that violate the rules)", You can design the presentation text to say, "The output is '1'."

[0063] To further improve the classification effectiveness of text classification models, A training policy that gradually increases the learning rate of the first model. A training policy that adjusts the model parameters of some layers of the first model, and A text classification model may be trained using at least one of the following training policies: training policies that set different learning rates for different layers of the text classification model.

[0064] In the training phase of a text classification model, a learning rate warm-up policy generally means using a relatively small learning rate at the start of training and gradually increasing the learning rate of the first model within a certain number of iterations (steps) or epochs until a predetermined frequency threshold (i.e., a predetermined relatively large value) is reached. This process helps the text classification model stabilize in the initial stages and avoids the training instability caused by a relatively large learning rate. The learning rate of the first model may be gradually increased depending on the linear method, or it may be gradually increased depending on the exponential method.

[0065] By using a warm-up training policy, it is possible to accelerate the convergence speed of a text classification model and further improve its classification accuracy.

[0066] Furthermore, after the warm-up phase is complete, the learning rate of the first model may be decayed according to a certain policy, such as step decay, exponential decay, or cosine annealing. These policies help the text classification model fine-tune its parameters more precisely during the mid-to-late stages of training, thereby leading to better convergence.

[0067] Adjusting the model parameters of the first model may destroy the knowledge it learned during the pre-training phase. Therefore, during the training process of the first model, it is acceptable to adjust only the model parameters of some layers of the first model. For example, only the model parameters of the last few layers of the first model may be adjusted. This balances the generalization function and knowledge utilization rate of the text classification model, preventing the first model from forgetting knowledge and thereby strengthening the classification function of the text classification model.

[0068] Different layers of the first model (i.e., a pre-trained large-scale language model) may acquire language features of different granularity. For example, lower layers may acquire local grammatical and lexical features, while higher layers may acquire more abstract semantic information. By setting different learning rates for different layers of the first model, each layer of the first model can be adjusted according to its learning needs to adapt to those needs. Additionally, some difficult-to-learn features may need to be updated effectively using a larger learning rate, while well-learned features may only require a smaller learning rate. Setting different learning rates for different layers of a text classification model can improve the training efficiency of the model. Furthermore, parameters relevant to the current classification task (e.g., activation function layer, first perfect connection layer, second perfect connection layer parameters) may need to be rapidly adapted to the current task using a larger learning rate, while the general-purpose language representation layer (i.e., the first model) needs to be maintained stable using a smaller learning rate. Setting different learning rates for different layers of a text classification model can accelerate the learning of task-relevant parameters and fine-tuning of general representations, thereby reducing the risk of overfitting. By using a small learning rate on a pre-trained first model, we can prevent the first model from overfitting on specific tasks, thereby improving its generalization ability to data it has not yet encountered.

[0069] Figure 5 is a flowchart of a text classification method according to an exemplary embodiment. As shown in Figure 5, the text classification method may include the following S201 to S203.

[0070] S201 retrieves the first text to be classified.

[0071] In S202, the first text is input to the first model of the target text classification model, and the second embedding vector matrix of the first generated word output from the first model is obtained. The target text classification model includes the first model and an activation function layer, and the first model is a large-scale language model.

[0072] In S203, the second embedding vector matrix is ​​input to the activation function layer, and the classification result of the first text output from the activation function layer is obtained.

[0073] In this disclosure, the target text classification model may be obtained by training it using the text classification model training method described herein.

[0074] In this embodiment, the target text classification model includes a first model and an activation function layer. The first model is a large-scale language model, and since the large-scale language model is applied to the processing of long texts and can acquire long-range contextual features, the target text classification model can support the classification of long texts. This allows the target text classification model to acquire long-range contextual features even for long texts, enabling attention to contextual features and thereby improving the classification effect of long texts. Furthermore, by using the second embedding vector matrix of the first generated word output from the first model as the input to the activation function layer, the input to the activation function layer can be made simpler and more controllable, thereby ensuring controllability of the output content of the target text classification model and further improving the text classification effect.

[0075] The following describes in detail a specific embodiment in which the first text in S202 above is input to the first model of the target text classification model, and the second embedding vector matrix of the first generated word output from the first model is obtained. Specifically, this may be implemented by multiple embodiments, and in one embodiment, it may be implemented by the following steps (b1) to (b3).

[0076] Step (b1) Obtain the presented text.

[0077] In this disclosure, the presented text used in the reasoning phase of the target text classification model matches the classification text used when training the target text classification model.

[0078] Step (b2) The presented text and the first text are combined to obtain the second combined text, and the second combined text is word-segmented to obtain the second word-segmentation result.

[0079] In this disclosure, the presented text and the first text may be joined in an order in which the presented text comes first and the first text comes second, or the presented text and the first text may be joined in an order in which the first text comes first and the presented text comes second. This disclosure does not specifically limit the order in which the presented text and the first text are joined, as long as the order in which the presented text and the text sample are joined matches the order in which the presented text and the first text are joined.

[0080] Step (b3) Input the second word segmentation result into the first model of the target text classification model to obtain the second embedding vector matrix.

[0081] In another embodiment, as shown in Figure 6, the above text classification method may further include S204 and S205 below.

[0082] In S204, once the target text classification model has finished loading, the presented text is retrieved, and the presented text is input into the first model to obtain the key-value matrix corresponding to the presented text.

[0083] In this disclosure, the key-value matrix is ​​the key-value matrix of the presented text, i.e., the k matrix and the v matrix.

[0084] In S205, the key-value matrix is ​​loaded into the first model.

[0085] In this case, inputting the first text into the first model of the target text classification model and obtaining the second embedding vector matrix of the first generated word output from the first model is: The first text is segmented into words, and the result of the third word segmentation is obtained. This may include inputting the third word segmentation result into the first model obtained by loading the key-value matrix to obtain a second embedding vector matrix.

[0086] In the above embodiment, a key-value cache strategy is used to pre-determine the key-value matrix of the presented text and pre-load it into the first model before the target text classification model infers its reasoning. In this way, only the first text needs to be input into the model during the reasoning stage of the target text classification model, eliminating the need to input the presented text. This reduces the iterative calculation of the k and v matrices of the presented text, thereby shortening the model's reasoning time and improving the model's reasoning efficiency.

[0087] In a selective embodiment, the target text classification model further includes a first fully connected layer, a coupled layer, and a second fully connected layer. The above text classification method is, This may further include obtaining the second statistical feature of the first text.

[0088] In this case, inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer is possible. The first fully connected layer increases the dimensionality of the second statistical feature to the size of the second embedding vector matrix, The second combined feature is obtained by combining the second statistical feature, which has been obtained by increasing its dimensionality through a combined layer, with the second embedding vector matrix. The second fully connected layer fuses the second connected feature to obtain the second fused feature, This may include generating a classification result of the first text based on a second fusion feature using an activation function layer.

[0089] In a selective embodiment, the second statistical feature includes at least one of text length, percentage of different language texts, percentage of digits, percentage of common punctuation marks, and percentage of special symbols.

[0090] Figure 7 is a block diagram of a text classification model training device according to an exemplary embodiment. The text classification model includes a first model and an activation function layer, the first model being a pre-trained large-scale language model, and as shown in Figure 7, the text classification model training device 300 is This is for acquiring a training sample set and a presentation text, wherein the training sample in the training sample set includes a text sample and a classification label corresponding to the text sample, and the presentation text includes classification task description information of the text classification model, and the first acquisition module 301 The system may include a training module 302 for training the text classification model to obtain a target text classification model by taking the presented text and the text sample as inputs to the first model, the first embedding vector matrix of the first generated word output from the first model as inputs to the activation function layer, and the classification label corresponding to the text sample as the target output of the activation function layer.

[0091] In the above technical solution, the text classification model includes a first model and an activation function layer. The first model is a pre-trained large-scale language model. Since the large-scale language model is applied to the processing of long texts and can acquire long-range contextual features, the text classification model can support the classification of long texts. The trained target text classification model has the ability to acquire long-range contextual features even for long texts, allowing it to pay attention to contextual features and thereby improving the classification effect of long texts. Furthermore, the large-scale language model has a larger amount of pre-trained language material, allowing it to compress more knowledge into the model. Combined with presentation word engineering for optimization, this significantly improves the classification function of the target text classification model. Additionally, by using the first embedding vector matrix of the first generated word output from the first model as the input to the activation function layer, the input to the activation function layer can be made simpler and more controllable, thereby ensuring controllability of the output content of the text classification model and further improving the model's classification function.

[0092] As an option, the text classification model further includes a first fully connected layer, a coupled layer, and a second fully connected layer. The aforementioned text classification model training device 300 is The system further comprises a second acquisition module for obtaining the first statistical features of the aforementioned text sample, The first model generates the first embedding vector matrix based on the presented text and the text sample, and inputs the first embedding vector matrix to the coupling layer. The first fully connected layer is for increasing the dimensionality of the first statistical feature to the size of the first embedding vector matrix, and for inputting the first statistical feature obtained with increased dimensionality into the connected layer. The aforementioned coupling layer is for obtaining a first combined feature by combining the first statistical feature obtained with the aforementioned dimensionality increase with the first embedding vector matrix, and for inputting the first combined feature to the second fully connected layer. The second fully connected layer is for obtaining a first fused feature by feature fusion of the first connected feature and for inputting the first fused feature to the activation function layer. The activation function layer is for generating the classification result of the text sample based on the first fusion feature.

[0093] As an option, the training module 302 is designed to train the text classification model and obtain a target text classification model by taking the presented text and the text sample as inputs to the first model, the first statistical features as inputs to the first fully connected layer, the first embedding vector matrix of the first generated word output from the first model and the output of the first fully connected layer as inputs to the connected layer, the output of the connected layer as inputs to the second fully connected layer, the output of the second fully connected layer as inputs to the activation function layer, and the classification label corresponding to the text sample as the target output of the activation function layer.

[0094] As an option, the first statistical feature includes at least one of the following: text length, percentage of different language texts, percentage of digits, percentage of common punctuation marks, and percentage of special symbols.

[0095] As an option, the presented text further includes illustrative information to guide the text classification model to output a classification label.

[0096] As an option, A training policy that gradually increases the learning rate of the first model described above. A training policy that adjusts the model parameters of some layers of the aforementioned first model, and The text classification model is trained using at least one of the training policies that set different learning rates for different layers of the text classification model.

[0097] As an option, the training module 302 is, A first combining submodule for obtaining a first combined text by combining the aforementioned presented text and the aforementioned text sample, and for obtaining a first word segmentation result by word segmenting the first combined text, The system includes a decision submodule for taking the first word segmentation result as input to the first model.

[0098] Figure 8 is a block diagram of a text classification device according to an exemplary embodiment. As shown in Figure 8, the text classification device 400 is: A third acquisition module 401 for obtaining the first text to be classified, The first text is input to the first model of the target text classification model to obtain the second embedding vector matrix of the first generated word output from the first model, and the feature extraction module 402 includes the first model and an activation function layer, and the first model is a large-scale language model, The system includes a classification module 403 for inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer.

[0099] In this embodiment, the target text classification model includes a first model and an activation function layer. The first model is a large-scale language model, and since the large-scale language model is applied to the processing of long texts and can acquire long-range contextual features, the target text classification model can support the classification of long texts. This allows the target text classification model to acquire long-range contextual features for long texts as well, enabling attention to contextual features and thereby improving the classification effect of long texts. Furthermore, by using the second embedding vector matrix of the first generated word output from the first model as the input to the activation function layer, the input to the activation function layer can be made simpler and more controllable, thereby ensuring controllability of the output content of the target text classification model and further improving the text classification effect.

[0100] As an option, the feature extraction module 402 is: A GET submodule for obtaining the displayed text, A second combine submodule for obtaining a second combined text by combining the aforementioned presented text and the first text, and for obtaining a second word segmentation result by word segmenting the second combined text, The system includes a first feature extraction submodule for inputting the second word segmentation result into the first model of the target text classification model to obtain the second embedding vector matrix.

[0101] As an option, the text classification device 400 is When the loading of the target text classification model is complete, a fourth acquisition module is provided to acquire the presented text and input the presented text into the first model to obtain a key-value matrix corresponding to the presented text. The model further comprises a load module for loading the aforementioned key-value matrix into the aforementioned first model, The feature extraction module 402 is, A word segmentation submodule for word segmenting the aforementioned first text and obtaining the result of a third word segmentation, The system includes a second feature extraction submodule for inputting the third word segmentation result into a first model obtained by loading a key-value matrix, in order to obtain the second embedding vector matrix.

[0102] As an option, the target text classification model further includes a first fully connected layer, a connected layer, and a second fully connected layer. The text classification device 400 is The system further comprises a fifth acquisition module for acquiring a second statistical feature of the first text, The aforementioned classification module 403 is A dimensionality increasing submodule for increasing the dimensionality of the second statistical feature to the size of the second embedding vector matrix by the first fully connected layer, A third coupling submodule for obtaining a second coupled feature by combining the second statistical feature obtained by increasing the dimensionality through the coupling layer with the second embedding vector matrix, A feature fusion submodule for obtaining a second fused feature by feature fusion of the second coupled feature with the second fully connected layer, The system includes a classification submodule for generating a classification result of the first text based on the second fusion feature using the activation function layer.

[0103] As an option, the second statistical feature includes at least one of the following: text length, percentage of different language texts, percentage of digits, percentage of common punctuation marks, and percentage of special symbols.

[0104] Furthermore, the Disclosure further provides a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processing unit, it enables the steps of the text classification model training method or the text classification method according to the Disclosure.

[0105] This disclosure further provides electronic devices, which are, A memory device in which a computer program is stored, The device comprises a processing unit for executing the computer program in the storage device to realize the steps of the text classification model training method relating to this disclosure, or the steps of the text classification method relating to this disclosure.

[0106] This disclosure further provides a computer program product, including a computer program, which, when executed by a processor, enables the steps of the text classification model training method or the text classification method described herein.

[0107] Hereinafter, with reference to Figure 9, a schematic diagram of the structure of an electronic device (terminal device or server) 600 suitable for realizing the embodiments of this disclosure is shown. The terminal devices in the embodiments of this disclosure may include, but are not limited to, mobile devices such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 9 is merely an example and does not in any way limit the functions and scope of use of the embodiments of this disclosure.

[0108] As shown in Figure 9, the electronic device 600 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 601, which can perform various appropriate operations and processes based on a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data necessary for the operation of the electronic device 600 are further stored in the RAM 603. The processing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0109] Generally, input devices 606, including, for example, touch panels, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 607, including, for example, liquid crystal displays (LCDs), speakers, and vibrators; storage devices 608, including, for example, magnetic tapes and hard disks; and communication devices 609 may be connected to the I / O interface 605. The communication devices 609 may be permitted to exchange data between the electronic device 600 and other devices via wireless or wired communication. Figure 9 shows an electronic device 600 with various devices, but it should be understood that it is not required to implement or have all of the illustrated devices, and more or fewer devices may be implemented or have been provided as alternatives.

[0110] In particular, according to embodiments of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program contained in a non-temporary computer-readable medium, and which includes program code for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network by a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, it performs the functions limited to the methods of embodiments of the present disclosure.

[0111] The computer-readable media described herein may be a computer-readable signal medium, a computer-readable storage medium, or a combination of both. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium may contain data signals propagating in the baseband or as part of a carrier, which may contain computer-readable program code. Such propagating data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may transmit, propagate, or transmit programs used by or in combination with instruction execution systems, apparatus, or devices. The program code contained in the computer-readable medium may be transmitted by any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), or any suitable combination thereof.

[0112] In some embodiments, client terminals and servers may communicate using any known or future-to-be-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and may be interconnected with digital data communications (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any known or future-to-be-developed networks.

[0113] The computer-readable medium described above may be included in the electronic device described above, or it may exist independently and not be assembled into the electronic device.

[0114] If the above-mentioned computer-readable medium contains one or more programs, and the above-mentioned one or more programs are executed by the electronic device, the electronic device is instructed to acquire a training sample set and a presentation text, that the training samples in the training sample set include text samples and classification labels corresponding to the text samples, that the presentation text includes classification task description information for a text classification model, that the text classification model includes a first model and an activation function layer, that the first model is a pre-trained large-scale language model, and that the presentation text and the text samples are inputs to the first model, that the first embedding vector matrix of the first generated word output from the first model is input to the activation function layer, and that the classification labels corresponding to the text samples are the target output of the activation function layer, thereby training the text classification model and obtaining a target text classification model.

[0115] Alternatively, if the computer-readable medium contains one or more programs and the one or more programs are executed by the electronic device, the electronic device is instructed to: obtain a first text to be classified; input the first text into a first model of a target text classification model to obtain a second embedding vector matrix of the first generated word output from the first model; confirm that the target text classification model includes the first model and an activation function layer, and that the first model is a large-scale language model; and input the second embedding vector matrix into the activation function layer to obtain the classification result of the first text output from the activation function layer.

[0116] Computer program code for performing the operations of the Disclosure may be organized using one or more programming languages ​​or a combination thereof, the programming languages ​​including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, and further including ordinary procedural programming languages ​​such as the "C" language or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, run as a single standalone software pack, run partly on the user's computer and partly on a remote computer, or run entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, connected via the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the drawings illustrate the implementable systematic architecture, functions, and operations of the systems, methods, and computer program products relating to each embodiment of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing a predetermined logical function. In alternative partial implementations, the functions described in the blocks may occur in an order different from that shown in the drawings. For example, two blocks shown consecutively may actually be executed almost in parallel or in reverse order, depending on the function. Each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented using a dedicated hardware-based system for performing a predetermined function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The modules described in the embodiments of this disclosure may be implemented in software or in hardware. The names of the modules may not be limiting to the modules themselves; for example, the first acquisition module may be further described as a "module for acquiring training sample sets and presentation texts."

[0119] The functions described herein may be performed by at least partially one or more hardware logic components. For example, exemplary types of hardware logic components used include, but are not limited to, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

[0120] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program used in or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wired electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0121] According to one or more embodiments of the present disclosure, Example 1 provides a text classification model training method, wherein the text classification model comprises a first model and an activation function layer, the first model being a pre-trained large-scale language model, and the text classification model training method is A training sample set and a presentation text are obtained, and the training samples in the training sample set include text samples and classification labels corresponding to the text samples, and the presentation text includes classification task description information of the text classification model, The method includes training the text classification model to obtain a target text classification model by using the presented text and the text sample as inputs to the first model, using the first embedding vector matrix of the first generated word output from the first model as input to the activation function layer, and using the classification label corresponding to the text sample as the target output of the activation function layer.

[0122] According to one or more embodiments of the present disclosure, Example 2 provides a method for training the text classification model of Example 1, wherein the text classification model further comprises a first fully connected layer, a coupled layer and a second fully connected layer. The aforementioned text classification model training method is: This further includes obtaining a first statistical feature of the aforementioned text sample, The first model generates the first embedding vector matrix based on the presented text and the text sample, and inputs the first embedding vector matrix to the coupling layer. The first fully connected layer is for increasing the dimensionality of the first statistical feature to the size of the first embedding vector matrix, and for inputting the first statistical feature obtained with increased dimensionality into the connected layer. The coupling layer is for obtaining a first combined feature by combining the first statistical feature obtained with increased dimensionality and the first embedding vector matrix, and for inputting the first combined feature to the second fully connected layer. The second fully connected layer is for obtaining a first fused feature by feature fusing the first connected feature and inputting the first fused feature to the activation function layer, 4 The activation function layer is for generating the classification result of the text sample based on the first fusion feature.

[0123] According to one or more embodiments of this disclosure, Example 3 provides the text classification model training method of Example 2, wherein the presented text and the text sample are input to the first model, the first embedding vector matrix of the first generated word output from the first model is input to the activation function layer, and the classification label corresponding to the text sample is the target output of the activation function layer, thereby training the text classification model and obtaining a target text classification model. The method includes training the text classification model to obtain a target text classification model by using the presented text and the text sample as inputs to the first model, the first statistical features as inputs to the first fully connected layer, the first embedding vector matrix of the first generated word output from the first model and the output of the first fully connected layer as inputs to the connected layer, the output of the connected layer as inputs to the second fully connected layer, the output of the second fully connected layer as inputs to the activation function layer, and the classification label corresponding to the text sample as the target output of the activation function layer.

[0124] According to one or more embodiments of the present disclosure, Example 4 provides the text classification model training method of Example 2, wherein the first statistical feature includes at least one of text length, percentage of different language text, percentage of digits, percentage of common punctuation, and percentage of special characters.

[0125] According to one or more embodiments of the present disclosure, Example 5 provides a text classification model training method described in any one of Examples 1 to 4, wherein the presented text further includes illustrative information to guide the text classification model to output classification labels.

[0126] According to one or more embodiments of this disclosure, Example 6 provides a text classification model training method described in any one of Examples 1 to 4. A training policy that gradually increases the learning rate of the first model described above. A training policy that adjusts the model parameters of some layers of the aforementioned first model, and The text classification model is trained using at least one of the training policies that set different learning rates for different layers of the text classification model.

[0127] According to one or more embodiments of this disclosure, Example 7 provides a text classification model training method described in any one of Examples 1 to 4, wherein the presented text and the text sample are used as inputs to the first model. The process involves combining the presented text and the text sample to obtain a first combined text, and then word-segmenting the first combined text to obtain a first word-segmentation result. This includes using the first word segmentation result as input to the first model.

[0128] According to one or more embodiments of the present disclosure, Example 8 provides a text classification method, said text classification method is To obtain the first text to be classified, The first text is input to the first model of the target text classification model, and the second embedding vector matrix of the first generated word output from the first model is obtained, and the target text classification model includes the first model and an activation function layer, and the first model is a large-scale language model, This includes inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer.

[0129] According to one or more embodiments of the present disclosure, Example 9 provides the text classification method of Example 8, in which the first text is input into a first model of a target text classification model to obtain a second embedding vector matrix of the first generated word output from the first model, To obtain the displayed text, The process involves combining the presented text and the first text to obtain a second combined text, and then word-segmenting the second combined text to obtain the second word-segmentation result. This includes inputting the second word segmentation result into the first model of the target text classification model to obtain the second embedding vector matrix.

[0130] According to one or more embodiments of this disclosure, Example 10 provides the text classification method of Example 8, the text classification method is When the loading of the target text classification model is complete, the presented text is retrieved, and the presented text is input into the first model to obtain a key-value matrix corresponding to the presented text. The method further includes loading the aforementioned key-value matrix into the aforementioned first model, Inputting the first text into the first model of the target text classification model and obtaining the second embedding vector matrix of the first generated word output from the first model is: The above first text is divided into words to obtain the result of the third word division, This includes inputting the third word segmentation result into a first model obtained by loading a key-value matrix to obtain the second embedding vector matrix.

[0131] According to one or more embodiments of the present disclosure, Example 11 provides a text classification method described in any one of Examples 8 to 10, wherein the target text classification model further comprises a first fully connected layer, a coupled layer and a second fully connected layer. The aforementioned text classification method is, Further including obtaining the second statistical feature of the first text, Inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer is: The first fully connected layer increases the dimensionality of the second statistical feature to the size of the second embedding vector matrix, The second combined feature is obtained by combining the second statistical feature, which is obtained by increasing the dimensionality through the aforementioned combined layer, with the second embedding vector matrix. The two coupled features are fused by the second fully connected layer to obtain a second fused feature, This includes generating a classification result for the first text based on the second fusion feature using the activation function layer.

[0132] According to one or more embodiments of the present disclosure, Example 12 provides the text classification method of Example 11, wherein the second statistical feature includes at least one of text length, percentage of different language text, percentage of digits, percentage of common punctuation, and percentage of special characters.

[0133] According to one or more embodiments of the present disclosure, Example 13 provides a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processing unit, it realizes a step of the text classification model training method described in any one of Examples 1 to 7, or a step of the text classification method described in any one of Examples 8 to 12.

[0134] According to one or more embodiments of this disclosure, Example 14 provides an electronic device, which is: A memory device in which a computer program is stored, The device comprises a processing unit for executing the computer program in the storage device to implement the steps of the text classification model training method described in any one of Examples 1 to 7, or the steps of the text classification method described in any one of Examples 8 to 12.

[0135] According to one or more embodiments of the present disclosure, Example 15 provides a computer program product including a computer program, which, when executed by a processor, realizes the steps of the text classification model training method described in any one of Examples 1 to 7, or the steps of the text classification method described in any one of Examples 8 to 12.

[0136] The above description is merely an explanation of preferred embodiments and the technical principles in which the present disclosure is applied. Those skilled in the art will understand that the scope of the disclosure is not limited to technical solutions made by specific combinations of the above technical features, but should also include other technical solutions made by arbitrarily combining the above technical features or equivalent features without departing from the above-disclosed concept. For example, technical solutions made by substituting the above features with technical features having similar functions to those disclosed in this disclosure (but not limited to those disclosed therein).

[0137] Furthermore, although each operation is described in a specific procedure, it should not be understood that these operations must be performed in the specific procedure or sequential execution shown. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above description, these should not be construed as limiting the scope of this disclosure. Several features described before and after a single embodiment may be combined and implemented in a single embodiment. Conversely, each feature described before and after a single embodiment may be implemented in multiple embodiments, either individually or in any suitable subcombination.

[0138] Although this subject matter has been described using terminology specific to structural features and / or methodological logic, it should be understood that the subject matter limited by the attached claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely exemplary forms of realizing the claims. The specific manner in which each module performs operations in the apparatus in the above embodiment has been described in detail in the embodiment relating to the method and will not be described in detail here.

Claims

1. A method for training a text classification model, The text classification model includes a first model and an activation function layer, wherein the first model is a pre-trained large-scale language model, and the training method for the text classification model is: A training sample set and a presentation text are obtained, and the training samples in the training sample set include text samples and classification labels corresponding to the text samples, and the presentation text includes classification task description information of the text classification model, A method for training a text classification model, characterized by: using the presented text and the text sample as inputs to the first model; using the first embedding vector matrix of the first generated word output from the first model as input to the activation function layer; and using the classification label corresponding to the text sample as the target output of the activation function layer to train the text classification model and obtain a target text classification model.

2. The text classification model further includes a first fully connected layer, a coupling layer, and a second fully connected layer. The aforementioned text classification model training method is: This further includes obtaining a first statistical feature of the aforementioned text sample, The first model generates the first embedding vector matrix based on the presented text and the text sample, and inputs the first embedding vector matrix to the coupling layer. The first fully connected layer increases the dimension of the first statistical feature to the size of the first embedding vector matrix, and inputs the obtained first statistical feature with increased dimension to the connected layer. The coupling layer is for obtaining a first combined feature by combining the first statistical feature obtained with increased dimensionality and the first embedding vector matrix, and for inputting the first combined feature to the second fully connected layer. The second fully connected layer is for obtaining a first fused feature by feature fusion of the first connected feature and for inputting the first fused feature to the activation function layer. The text classification model training method according to claim 1, characterized in that the activation function layer is for generating classification results of the text samples based on the first fused features.

3. By using the presented text and the text sample as inputs to the first model, the first embedding vector matrix of the first generated word output from the first model as input to the activation function layer, and the classification label corresponding to the text sample as the target output of the activation function layer, the text classification model can be trained to obtain a target text classification model. The text classification model training method according to claim 2, characterized in that the presented text and the text sample are inputs to the first model, the first statistical features are inputs to the first fully connected layer, the first embedding vector matrix of the first generated word output from the first model and the output of the first fully connected layer are inputs to the connected layer, the output of the connected layer is input to the second fully connected layer, the output of the second fully connected layer is input to the activation function layer, and the classification label corresponding to the text sample is the target output of the activation function layer, thereby training the text classification model to obtain a target text classification model.

4. The text classification model training method according to claim 2, characterized in that the first statistical feature includes at least one of text length, the proportion of different language texts, the proportion of numbers, the proportion of commonly used punctuation marks, and the proportion of special symbols.

5. The text classification model training method according to any one of claims 1 to 4, characterized in that the presented text further includes illustrative information for guiding the text classification model to output classification labels.

6. A training policy that gradually increases the learning rate of the first model, A training policy that adjusts the model parameters of some layers of the first model, and A method for training a text classification model according to any one of claims 1 to 4, characterized in that the text classification model is trained using at least one of the training policies that set different learning rates for different layers of the text classification model.

7. Using the aforementioned presented text and the aforementioned text sample as input to the first model means that The present text and the text sample are combined to obtain a first combined text, and the first combined text is word-segmented to obtain a first word-segmentation result. A text classification model training method according to any one of claims 1 to 4, characterized in that the first word segmentation result is used as input to the first model.

8. A text classification method, To obtain the first text to be classified, The first text is input to the first model of the target text classification model, and the second embedding vector matrix of the first generated word output from the first model is obtained, and the target text classification model includes the first model and an activation function layer, and the first model is a large-scale language model, A text classification method characterized by comprising inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer.

9. Inputting the first text into the first model of the target text classification model and obtaining the second embedding vector matrix of the first generated word output from the first model is: To obtain the displayed text, The process involves combining the presented text and the first text to obtain a second combined text, and then performing word segmentation on the second combined text to obtain the second word segmentation result. The text classification method according to claim 8, characterized in that it includes inputting the second word segmentation result into the first model of the target text classification model to obtain the second embedding vector matrix.

10. The aforementioned text classification method is, When the loading of the target text classification model is complete, the presented text is retrieved, and the presented text is input into the first model to obtain a key-value matrix corresponding to the presented text. The method further includes loading the key-value matrix into the first model, Inputting the first text into the first model of the target text classification model and obtaining the second embedding vector matrix of the first generated word output from the first model is: The first text is segmented into words, and the result of the third word segmentation is obtained. The text classification method according to claim 8, characterized in that it includes inputting the third word segmentation result into a first model obtained by loading a key-value matrix to obtain the second embedding vector matrix.

11. The aforementioned target text classification model further includes a first fully connected layer, a coupled layer, and a second fully connected layer. The aforementioned text classification method is, Further including obtaining the second statistical feature of the first text, Inputting the second embedding vector matrix into the activation function layer and obtaining the classification result of the first text output from the activation function layer is: The first fully connected layer increases the dimensionality of the second statistical feature to the size of the second embedding vector matrix, The second statistical feature obtained by increasing the dimensionality using the aforementioned coupling layer is combined with the second embedding vector matrix to obtain a second combined feature, The two coupled features are fused by the two fully connected layers to obtain a second fused feature, The text classification method according to any one of claims 8 to 10, characterized in that it includes generating a classification result of the first text based on the second fusion feature using the activation function layer.

12. The text classification method according to claim 11, characterized in that the second statistical feature includes at least one of text length, the proportion of different language texts, the proportion of numbers, the proportion of commonly used punctuation marks, and the proportion of special symbols.

13. A computer-readable medium on which computer programs are stored, A computer-readable medium characterized in that, when the computer program is executed by a processing unit, it realizes the steps of the text classification model training method described in any one of claims 1 to 4, or the steps of the text classification method described in any one of claims 8 to 10.

14. It is an electronic device, A memory device in which a computer program is stored, An electronic device comprising: a processing unit for executing the computer program in the storage device so as to realize the steps of the text classification model training method described in any one of claims 1 to 4, or the steps of the text classification method described in any one of claims 8 to 10.

15. It is a computer program, A computer program that, when executed by a processor, causes the processor to perform the steps of the text classification model training method described in any one of claims 1 to 4, or the steps of the text classification method described in any one of claims 8 to 10.

Images