Text classification method and device, nonvolatile storage medium and electronic equipment

By introducing adaptive network layer and attention mechanism module into the text classification model, combining feedforward neural network and sub-learning module, the problem of unsatisfactory text classification accuracy caused by insufficient training resources of big model is solved, and the problem of unsatisfactory text classification accuracy under limited resources is realized. Efficient text classification.

CN118916485APending Publication Date: 2024-11-08CHINA TELECOM BESTPAY CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410863784.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-29
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

During the training and deployment of existing large models, due to the high computing resource consumption, the accuracy of text classification is not ideal, especially under the conditions of limited training resources, it is difficult to achieve ideal classification effects.

Method used

Adaptive network layer is adopted, by including multiple network layers in the target model and connecting the adaptive network separately, the parameters in the adaptive network are adjusted to complete model training, and combined with the attention mechanism module and the feedforward neural network module, a predetermined replacement is adopted. Strategy and sub-learning modules to extract and process semantic dependency features, reduce model complexity and improve training efficiency.

Benefits of technology

Improve the accuracy of text classification under limited training resources, reduce the amount of parameters required for model training, maintain classification accuracy, and alleviate the problems of high computing costs and inefficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916485A_ABST
    Figure CN118916485A_ABST
Patent Text Reader

Abstract

The invention discloses a text classification method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps that processing is conducted on the basis of an input text vector and a plurality of network layers included in an input target model, semantic dependency features are determined, the network layers are connected with self-adaptive networks respectively, and the parameter quantity of the self-adaptive networks is smaller than that of the corresponding network layers; the target model is used for completing model training processing by adopting a mode of adjusting parameters included in the adaptive network; and processing the semantic dependency features by adopting a classifier included in the target model to obtain a target classification result of the input text vector. The technical problem that the accuracy of text classification is not ideal under the condition that training resources are limited in the prior art is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a text classification method, device, non-volatile storage medium and electronic device. Background Art

[0002] With the continuous development of deep learning technology, it has been widely used in many fields due to its excellent performance. However, these large models usually contain a huge number of parameters, often hundreds of millions or even hundreds of billions, which leads to significant computing resource consumption and low training efficiency. This not only increases the cost of model training and deployment, but also limits the widespread use of models in real-world scenarios and industrial applications. Under the condition of limited training resources, related technologies have unsatisfactory text classification accuracy due to the mismatch between computing resource consumption and limited learning capabilities.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] Embodiments of the present invention provide a text classification method, device, non-volatile storage medium and electronic device to at least solve the technical problem in the related art that the accuracy of text classification is not ideal under the condition of limited training resources.

[0005] According to one aspect of an embodiment of the present invention, a text classification method is provided, comprising: based on an input text vector, a plurality of network layers included in an input target model are processed to determine semantic dependency features, wherein the plurality of network layers are respectively connected to an adaptive network, the parameter amount of the adaptive network is smaller than the parameter amount of the corresponding network layer, and the target model completes model training processing by adjusting the parameters included in the adaptive network; and the semantic dependency features are processed by a classifier included in the target model to obtain a target classification result of the input text vector.

[0006] Optionally, the multiple network layers further include corresponding attention mechanism modules and feedforward neural network modules, respectively, and the attention mechanism module and the feedforward neural network module are connected through a residual module, wherein the attention mechanism module is used to extract features at different positions in the input text vector, and the feedforward neural network module is used to increase the nonlinear capability of the corresponding network layer; according to a predetermined replacement strategy, the adaptive network is connected to the corresponding network layer by replacing the attention mechanism module and / or the feedforward neural network module included in the corresponding network layer, wherein the multiple network layers are respectively set with predetermined replacement strategies.

[0007] Optionally, the multiple network layers are superimposed in a predetermined order, and the input text vector is inputted into the multiple network layers included in the target model for processing to determine the semantic dependency features, including: inputting the input text vector into the multiple network layers in sequence according to the predetermined order for processing; using the output features of the previous network layer included in the multiple network layers as the input features of the next network layer, and obtaining the output features of the last network layer included in the multiple network layers as the semantic dependency features, wherein the multiple network layers are used to extract text features at different levels respectively.

[0008] Optionally, the method of using the output features of a previous network layer included in the multiple network layers as input features of a subsequent network layer includes: performing dimensionality reduction processing based on the output features of the previous network layer to obtain reduced dimensionality features; performing nonlinear processing based on the reduced dimensionality features to obtain nonlinear features; when the adaptive network includes a sub-learning module, using the sub-learning module to perform feature extraction based on the nonlinear features to obtain target features, wherein the sub-learning module is used to enhance the ability to extract features; performing dimensionality increase processing based on the target features to obtain input features of the subsequent network layer.

[0009] Optionally, the sub-learning module is set to at least any one of the following: a self-attention layer, a convolutional layer, and a fully connected layer.

[0010] Optionally, the method further includes: when the adaptive network does not include a sub-learning module, performing dimensionality increase processing based on the nonlinear features to obtain input features of the subsequent network.

[0011] Optionally, the multiple network layers are superimposed in a predetermined order, and the multiple network layers are respectively provided with sub-classifiers. The method also includes: using the sub-classifiers respectively provided for the multiple network layers, classifying the output features respectively generated by the multiple network layers to obtain sub-classification results respectively corresponding to the multiple network layers; determining the consistency results between the sub-classification results respectively corresponding to a predetermined number of network layers among the multiple network layers from front to back according to the predetermined order indication; if the consistency result meets a predetermined condition, stopping the processing of the network layers after the predetermined number of network layers among the multiple network layers; and determining the target classification result based on the sub-classification results respectively corresponding to the predetermined number of network layers.

[0012] According to another aspect of an embodiment of the present invention, a text classification device is provided, comprising: an input module, for processing a plurality of network layers included in an input target model based on an input text vector to determine semantic dependency features, wherein the plurality of network layers are respectively connected to an adaptive network, a parameter amount of the adaptive network is smaller than a parameter amount of a corresponding network layer, and the target model completes model training processing by adjusting the parameters included in the adaptive network; and a classification module, for processing the semantic dependency features by using a classifier included in the target model to obtain a target classification result of the input text vector.

[0013] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing any one of the text classification methods.

[0014] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the text classification methods.

[0015] In an embodiment of the present invention, the semantic dependency features are determined by processing the multiple network layers included in the input target model based on the input text vector, wherein the multiple network layers are respectively connected to an adaptive network, the parameter amount of the adaptive network is less than the parameter amount of the corresponding network layer, and the target model completes the model training process by adjusting the parameters included in the adaptive network; the classifier included in the target model is used to process the semantic dependency features to obtain the target classification result of the input text vector. The purpose of improving the accuracy of text classification under limited training resources is achieved, the amount of parameters required for model training is reduced, and the technical effect of maintaining the accuracy of text classification is achieved, thereby solving the technical problem of unsatisfactory accuracy of text classification under the condition of limited training resources in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0017] Figure 1 is a flowchart of an optional text classification method provided according to an embodiment of the present invention;

[0018] Figure 2is a model schematic diagram of an optional text classification method provided according to an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of an optional text classification device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to an embodiment of the present invention, a method embodiment of text classification is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0023] Figure 1 is a flowchart of an optional text classification method provided according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0024] Step S102, based on the input text vector, multiple network layers included in the input target model are processed to determine semantic dependency features, wherein the multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is smaller than the parameter amount of the corresponding network layer, and the target model completes the model training process by adjusting the parameters included in the adaptive network;

[0025] It can be understood that the input text vector is processed through multiple network layers in the target model to determine the semantic dependency features. The semantic dependency features refer to the semantic relationship between words or phrases in the text, which provide data support for understanding the overall meaning of the text and making accurate classifications.

[0026] The above-mentioned multiple network layers do not work independently, but are connected to the adaptive network (also called Adapter network) respectively. The adaptive network is a predetermined neural network structure whose parameters are smaller than the corresponding network layer parameters. While maintaining a certain performance, the adaptive network can significantly reduce the complexity of the model, thereby reducing the demand for training resources. The training of the target model is completed by adjusting the parameters in the adaptive network. This method can avoid full training under limited training resources, find the optimal model parameters through the parameter iteration of the adaptive network, and alleviate the problem of high computational cost and low efficiency of the target model during training.

[0027] Optionally, the target model can be recorded as a PF-BERxiT framework, where multiple network layers can be recorded as Transformer units, and the adaptive network can be recorded as an Adapter module. The framework customizes the Adapter module that integrates the sub-learning module and connects it to each Transformer unit. During the training process of the target model, only the parameters of the Adapter module are fine-tuned, so that a better balance is achieved between the accuracy and efficiency of the trained target model.

[0028] In an optional embodiment, multiple network layers further include corresponding attention mechanism modules and feedforward neural network modules, respectively, and the attention mechanism module and the feedforward neural network module are connected through a residual module, wherein the attention mechanism module is used to extract features at different positions in the input text vector, and the feedforward neural network module is used to increase the nonlinear capability of the corresponding network layer; according to a predetermined replacement strategy, the adaptive network is connected to the corresponding network layer by replacing the attention mechanism module and / or the feedforward neural network module included in the corresponding network layer, wherein predetermined replacement strategies are set correspondingly for multiple network layers.

[0029] It can be understood that multiple network layers are responsible for processing the input text vector, and each network layer can be set to include a corresponding attention mechanism module and a feedforward neural network module. The main function of the attention mechanism module is to extract features at different positions in the input text vector. In natural language processing, different words or phrases often have different importance in the text. The attention mechanism can help the model identify and focus on these important parts, thereby improving the accuracy of classification. The feedforward neural network module is used to increase the nonlinear ability of the corresponding network layer. In natural language processing tasks, data often presents nonlinear relationships. The feedforward neural network can capture nonlinear relationships through nonlinear activation functions such as ReLU (Rectified Linear unit) and Sigmoid, which can further improve the performance of the model. In order to avoid gradient disappearance, the attention mechanism module and the feedforward neural network module can be connected through a residual module. The residual module allows the network layer to directly access the output of its previous layer, thereby effectively solving the gradient disappearance or gradient explosion problems that may occur in the deep network during training, and improving the training efficiency and performance of the network.

[0030] According to the predetermined replacement strategy, the adaptive network is connected to the corresponding network layer by replacing the attention mechanism module and / or the feedforward neural network module included in the corresponding network layer. The predetermined replacement strategy can be set according to the specific task requirements and network structure. For example, in some cases, only the attention mechanism module needs to be replaced, while in other cases, both the attention mechanism module and the feedforward neural network module need to be replaced. Each network layer can choose the most appropriate replacement method according to its own characteristics, so as to maximize the performance of the entire model.

[0031] The above processing is helpful to improve flexibility, so that the target model can adapt to different task requirements and achieve optimal performance with limited training resources.

[0032] Optionally, the target model is mainly composed of a superposition of multiple layers of Transformer units (i.e., multiple network layers). The output of the previous layer of Transformer units is the input of the next layer of Transformer units. Each Transformer unit includes a multi-head attention network (Multi-Head Attention) and a feedforward neural network (FFN). A residual connection is then used, and a normalization module can also be connected afterwards.

[0033] Optionally, the attention mechanism module Attn maps the query and key and value Represents the correlation matrix, which can be mathematically expressed as follows:

[0034]

[0035] Where n and m are the number of queries and key-value pairs, respectively.

[0036] Optionally, the above attention mechanism module can adopt a multi-head attention mechanism or a single-head attention mechanism. The multi-head attention module allows the model to use a single attention head to jointly focus on information at different positions from subspaces of different representations. For a single attention head, this feature is suppressed on average. The formula of the multi-head attention mechanism is denoted as MultiHead(Q, K, V). The function representation of the multi-head attention mechanism accepts three inputs: query (Query, denoted as Q), key (Key, denoted as K) and value (Value, denoted as V). As shown below:

[0037] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0038] head i =Attention(QW i Q ,KW i K ,VW i V )

[0039] Among them, head1,…,head h Represents the transformation matrices of h single-head attention modules, i-th query, key, and value respectively Among them, the final output transformation matrix is Concat represents a concatenation operation. It is preferred to use h = 8 parallel attention layers, d model Refers to the dimension of the vector in the model, where d k =d v =d model / h=64, specifies that 8 attention heads are used, and the key (d k ) and value (d v ) is 64. Due to the reduced size of each head, the total computational cost of the multi-head attention mechanism is similar to that of the single-head attention mechanism with full dimensions. i denotes the identifier of different network layers.

[0040] Optionally, the feedforward neural network module (FFN) can be set to a multi-layer perceptron network (MLP), which can be considered as an implementation of a fully connected feedforward network, consisting of two linear transformation layers with a ReLU (Rectified Linear unit) activation function in between:

[0041] FFN(x)=ReLU(xW1+b1)W2+b2

[0042] in, After that, residual connection and layer normalization are performed. The adaptive network (Adapter structure) is connected to the attention mechanism module and feedforward neural network module of each Transformer unit, and the adaptive network is trained in the process. x represents the output of a certain layer.

[0043] There are many ways to connect the adaptive network to the Transformer block. For example, it can be connected after the multi-head attention mechanism module or the feedforward neural network module, or in parallel with both, or even inserted after the residual layer (denoted as Add&Norm layer). The predetermined replacement strategy can also include different locations for the adaptive network connection, such as parallel setting (parallel adapter) and sequence setting (sequence adapter).

[0044] In an optional embodiment, multiple network layers are superimposed in a predetermined order, and based on the input text vector, the multiple network layers included in the input target model are processed to determine the semantic dependency features, including: inputting the input text vector into the multiple network layers in sequence according to the predetermined order for processing; using the output features of the previous network layer included in the multiple network layers as the input features of the next network layer, and obtaining the output features of the last network layer included in the multiple network layers as the semantic dependency features, wherein the multiple network layers are used to extract text features at different levels respectively.

[0045] It can be understood that multiple network layers are superimposed in a predetermined order so as to process the input text vector layer by layer and determine the semantic dependency features. The input text vector is input into the multiple network layers in the target model in a predetermined order for processing. When the input text vector passes through each network layer, the network layer will perform specific processing on the input features and generate output features, which are then passed to the next network layer as the input features of the next layer. It can be seen that the output features of the previous network layer are used as the input features of the next network layer. The layer-by-layer transmission method ensures the effective flow of information in the network, so that each network layer can use the processing results of all previous layers.

[0046] Each network layer is designed to extract text features at different levels. For example, a shallower network may focus on features at the vocabulary level, while a deeper network may focus more on features at the sentence or paragraph level. This hierarchical feature extraction method helps capture the complex structure and semantic relationships in the text. After the input text vector passes through all network layers, the output features of the last network layer are determined as semantic dependency features. The semantic dependency features contain the key information in the input text vector and can effectively reflect the semantic dependency relationships in the text vector.

[0047] Through the above processing, this layer-by-layer stacking and feature transfer method can effectively extract and utilize multi-level features in the text under limited computing resources, thereby improving the accuracy and efficiency of tasks such as text classification. At the same time, since each network layer can be customized and optimized as needed, the target model has high flexibility and scalability.

[0048] Optionally, the predetermined order may be set according to the functionality and complexity of the network layers, typically starting from a simpler feature extraction layer and gradually transitioning to a more complex feature fusion and conversion layer.

[0049] In an optional embodiment, the output features of a previous network layer included in multiple network layers are used as input features of a subsequent network layer, including: performing dimensionality reduction processing based on the output features of the previous network layer to obtain reduced dimensionality features; performing nonlinear processing based on the reduced dimensionality features to obtain nonlinear features; when a sub-learning module is included in the adaptive network, using the sub-learning module to perform feature extraction based on the nonlinear features to obtain target features, wherein the sub-learning module is used to enhance the ability to extract features; performing dimensionality increase processing based on the target features to obtain input features of the subsequent network layer.

[0050] It can be understood that in the process of feature transmission between multiple network layers, some intermediate processing steps are added to improve the network's processing ability and efficiency of features. When the output features of a network layer are passed to the next layer, the output features are first subjected to dimensionality reduction processing. The main purpose of dimensionality reduction processing is to reduce the dimension of the features, thereby reducing computational complexity and reducing the risk of overfitting. The features after dimensionality reduction processing will undergo a nonlinear processing step. Nonlinear processing is the key to the ability of neural networks to learn and represent complex patterns. It is achieved by applying activation functions (such as ReLU, Sigmoid, hyperbolic tangent activation function Tanh, etc.). The above activation functions can introduce nonlinear factors, allowing the network to capture the complex relationship between input and output.

[0051] If the adaptive network contains sub-learning modules, the features after non-linear processing will be further processed by this sub-learning module. To enhance the network's feature extraction ability for specific tasks, for example, it can be set as a convolutional layer dedicated to processing image details, or a recurrent neural network (RNN) or long short-term memory network (LSTM) for processing sequence data. The features processed by the sub-learning module (i.e., the target features) will be dimensionally upsampled to match the input dimension of the next network layer, aiming to ensure that the features can be correctly passed to the next layer and restore or enhance the feature representation ability when necessary. Through the above processing, the feature transfer between network layers is optimized, thereby improving the performance of the entire network, allowing the network to more effectively extract and transfer useful feature information while maintaining computational efficiency.

[0052] Optionally, there can be multiple dimensionality reduction methods, such as including principal components analysis (PCA) and autoencoder (AE).

[0053] Optionally, there can be multiple dimensionality upsampling processes, such as upsampling and interpolation.

[0054] Optionally, the architecture of the adaptive network can be a bottleneck architecture, which is a neural network structure. First, project the original d-dimensional input representation onto a low-dimensional r (r << d) through a lower projection matrix, then use a non-linear activation function g(.) and a sub-learning module f(.) to learn the hidden features in the input data, and then project the r-dimensional representation back to d dimensions through an upper projection matrix Let r be the dimension of the bottleneck, and let be the output of each adapter module (adaptive network). The formula for each adapter module is expressed as:

[0055]

[0056]

[0057]

[0057] g(x)=max(0,x)

[0058]

[0059] In an optional embodiment, the sub-learning module is set to at least any one of the following: a self-attention layer, a convolutional layer, and a fully connected layer.

[0060] It can be understood that the sub-learning module can be set to at least any one of the following three types: self-attention layer, convolutional layer and fully connected layer. The self-attention layer allows the model to pay different degrees of attention to different positions in the input sequence. In the self-attention layer, the output at each position depends on the input of all positions, and the attention weights are calculated to determine which positions to pay attention to. This allows the model to capture long-distance dependencies in the input data and is more effective when processing sequence data (such as text, time series, etc.).

[0061] The convolutional layer is the core component in the Convolutional Neural Networks (CNN), which extracts local features by applying convolution operations on the input data and captures local and global patterns in the data through sharing of convolution kernels and hierarchical feature extraction.

[0062] The fully connected layer achieves complete information transfer by connecting each input node to each output node. In the fully connected layer, each output node is a weighted sum of the input nodes and is transformed by a nonlinear activation function. The fully connected layer is usually used as the last layer in classification or regression tasks as a sub-learning module to enhance the representation ability of the network or introduce additional nonlinear transformations.

[0063] In an optional embodiment, the method further includes: when the sub-learning module is not included in the adaptive network, performing dimensionality increase processing based on nonlinear features to obtain input features of a subsequent network.

[0064] It is understandable that if the adaptive network does not include a sub-learning module, the dimension will be directly increased based on the features after nonlinear processing to obtain the input features of the next network layer. The network can still effectively transfer and transform features through the three steps of dimension reduction, nonlinear processing and dimension increase, and the risk of overfitting can be reduced by simplifying the network structure.

[0065] Optionally, to improve the classification efficiency of the target model, four different settings are added in the adapter module (i.e., the adaptive network) to distinguish different sub-learning modules f(.) to learn the hidden features of the samples, so as to enhance the model's ability to extract information from the hidden layer. These mainly include four ways of setting the adaptive network as PF-BERxiT(origin-adapter), PF-BERxiT(self-attn-adapter), PF-BERxiT(conv-adapter), and PF-BERxiT(mlp-adapter).

[0066] When the adaptive network is set to PF-BERxiT(origin-adapter), it only contains the lower projection matrix The original d-dimensional input representation is projected onto a low-dimensional r (r << d), followed by a non-linear activation function g(.), and then projected back to d dimensions through the upper projection matrix That is, there is no sub-learning module in the adapter structure (adaptive network). Under this condition, the formula of the adapter is:

[0067]

[0068] When the adaptive network is set to PF-BERxiT(self-attn-adapter), the sub-learning module is the self-attention mechanism layer, that is, it contains the lower projection matrix The original d-dimensional input representation is projected onto a low-dimensional r (r << d), followed by a non-linear activation function g(.) and the sub-learning module of the self-attention mechanism layer. Then, a non-linear activation function can be selectively added, which can be RELU, GELU, and Tanh, or the identity transformation. Finally, it is projected back to d dimensions through the upper projection matrix That is, there is a single-layer attention mechanism sub-learning module in the adapter structure. Under this condition, the formula of the adapter is:

[0069]

[0070] When the adaptive network is set to PF-BERxiT(self-attn-adapter), the sub-learning module is a one-dimensional convolutional layer, that is, it contains the lower projection matrix The original d-dimensional input representation Projected onto a low-dimensional space \(r (r\ll d)\), followed by a non-linear activation function \(g(.)\) and a one-dimensional convolutional layer in the sub-learning module. Then, a non-linear activation function can be selectively added, which can be RELU, GELU, Tanh, or an identity transformation. Finally, through the up-projection matrix Project the \(r\)-dimensional representation back to the \(d\)-dimensional space. That is, there is a one-dimensional convolutional neural network sub-learning module in the adapter structure. Under this condition, the formula of the adapter is:

[0071]

[0072] The adaptive network is set to PF-BERxiT (mlp-adapter), and the sub-learning module is a simple fully-connected layer, that is, it includes a down-projection matrix Project the original \(d\)-dimensional input representation Projected onto a low-dimensional space \(r (r\ll d)\), followed by a non-linear activation function \(g(.)\) and a simple fully-connected layer in the sub-learning module. Then, a non-linear activation function can be selectively added, which can be RELU, GELU, Tanh, or an identity transformation. Finally, through the up-projection matrix Project the \(r\)-dimensional representation back to the \(d\)-dimensional space. That is, there is a fully-connected sub-learning module in the adapter structure. Under this condition, the formula of the adapter is:

[0073]

[0074] In step S104, use the classifier included in the target model to process the semantic dependency features to obtain the target classification result of the input text vector.

[0075] It can be understood that using the classifier included in the target model to process these features to obtain the target classification result of the input text vector. Through the above processing, the accuracy of text classification can be improved with limited training resources. The use of the adaptive network reduces the complexity of the model and the requirements for computing resources and storage space. By adjusting the parameters in the adaptive network for model training, the generalization ability of the model can be improved while ensuring performance.

[0076] Optionally, the above classifier can be a classifier in various machine learning algorithms, such as support vector machines, random forests, neural networks, etc. Using the adaptive networks corresponding to multiple network layers to extract semantic dependency features, the classifier can obtain a more accurate target classification result based on the semantic dependency features.

[0077] In an optional embodiment, multiple network layers are superimposed in a predetermined order, and the multiple network layers are respectively provided with sub-classifiers. The method also includes: using the sub-classifiers respectively provided in the multiple network layers, classifying the output features respectively generated by the multiple network layers, and obtaining sub-classification results respectively corresponding to the multiple network layers; determining the consistency results between the sub-classification results respectively corresponding to a predetermined number of network layers in the multiple network layers from front to back according to the predetermined order indication; when the consistency result meets the predetermined condition, stopping the processing of the network layers after the predetermined number of network layers in the multiple network layers; and determining the target classification result based on the sub-classification results respectively corresponding to the predetermined number of network layers.

[0078] It can be understood that multiple network layers are stacked in a predetermined order, and each network layer receives the output of the previous layer as input and generates its own output features. Each network layer is equipped with a subclassifier for classifying the output features generated by the layer and generating corresponding subclassification results. In the process of passing through the network layer, each subclassifier will classify the output features generated by its corresponding network layer to obtain subclassification results. In a predetermined order, the consistency between the subclassification results corresponding to a predetermined number of network layers in the multiple network layers is checked from front to back. This usually involves comparing whether these subclassification results are the same or whether they meet certain similarity conditions. If the consistency result meets the predetermined condition, for example, multiple subclassification results are the same or the similarity reaches a certain threshold, the network layer after the predetermined number of network layers is stopped for further processing, which can save computing resources and speed up the classification process. Based on the subclassification results corresponding to the predetermined number of network layers, the final target classification result is determined.

[0079] Through the above processing, by setting a sub-classifier after each network layer, the target model can make predictions at an early stage of data delivery. When the consistency between the sub-classification results meets the predetermined conditions, the processing of the subsequent network layer can be stopped. This can significantly reduce the consumption of computing resources and time. Since each network layer is equipped with a sub-classifier, it is easier to analyze and explain the decision-making process of the network, which increases interpretability.

[0080] Optionally, the predetermined condition can be achieved by setting the number of levels of sub-classification result matching, or weighted averaging.

[0081] Through the above step S102, based on the input text vector, the input target model includes multiple network layers for processing to determine the semantic dependency features, wherein the multiple network layers are respectively connected to an adaptive network, the parameter amount of the adaptive network is less than the parameter amount of the corresponding network layer, and the target model is to complete the model training process by adjusting the parameters included in the adaptive network; step S104, the classifier included in the target model is used to process the semantic dependency features to obtain the target classification result of the input text vector. The purpose of improving the accuracy of text classification under limited training resources can be achieved, the amount of parameters that need to be processed in model training is reduced, and the technical effect of maintaining the accuracy of text classification is achieved, thereby solving the technical problem of unsatisfactory accuracy of text classification under the condition of limited training resources in the related art.

[0082] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation method, and the training method of the target model is described. During the training process, the Adam optimizer (an optimizer with adaptive learning rate) can be used to train the model, and the polynomial learning rate method is used to linearly decay the learning rate. During the training process, only the parameters of the adapter module (i.e., the adaptive network) are trained, thereby alleviating the problems of excessive computation and low efficiency in large Transformer models. In addition, the added sub-learning module is proven to be helpful in learning the hidden features of the sample and improving the performance of the model. In order to improve the performance of the parameter efficient transfer learning method, different sub-learning modules (including attention mechanism modules, one-dimensional convolution modules, and simple fully connected layers) are integrated into the adapter module, and the model performance is improved under limited training resources.

[0083] Figure 2 This is a model diagram of an optional text classification method provided according to an embodiment of the present invention, which is illustrated by a text classification task. The target model can be recorded as a PF-BERxiT framework, wherein multiple network layers can be recorded as Transformer units, and the adaptive network can be recorded as an Adapter module. The framework customizes the Adapter module that integrates the sub-learning module and is connected to each Transformer unit. In the training process of the target model, only the parameters of the Adapter module are fine-tuned, so that a better balance is achieved between the accuracy and efficiency of the trained target model.

[0084] The model training process includes the following steps:

[0085] Step S1: Connect an Adapter network to each Transformer unit. The Adapter can replace the attention mechanism layer or linear layer of each Transformer unit.

[0086] In step S2, the structure of the Adapter network mainly consists of a dimensionality reduction module, a nonlinear activation module, a sub-learning module, and a dimensionality increase module.

[0087] In step S3, the dimensionality reduction module of the Adapter network mainly maps high-dimensional data to low-dimensional data to reduce the training parameters of the model.

[0088] In step S4, the nonlinear module of the Adapter network mainly includes Tanh activation function, RELU activation function and GELU activation function, which are mainly used to increase the nonlinearity of the neural network model and improve the learning ability of the model.

[0089] Step S5: The sub-learning modules of the Adapter network include but are not limited to a single-layer attention mechanism model, a convolutional model, and a multi-layer perceptron model, which are used to enhance the model's ability to learn deep knowledge.

[0090] In step S6, the dimension-increasing module of the Adapter network mainly maps low-dimensional data to high-dimensional data to maintain the same dimension as the output of the next Transformer unit.

[0091] Step S7, during training, only the parameters of the adapter module are trained. This method can train fewer parameters and achieve the same effect as training all the parameters of the model, shortening the training time and saving computing resources.

[0092] The target model is mainly composed of multiple layers of Transformer units (i.e., multiple network layers). The output of the previous layer of Transformer units is the input of the next layer of Transformer units. Each Transformer unit includes a multi-head attention mechanism network (Multi-Head Attn) and a feed-forward neural network (FFN). After that, a residual connection is used, and a normalization module can also be connected afterwards.

[0093] Attention mechanism module Attn mapping query and key-value pairs as well as Represents the correlation matrix, which can be mathematically expressed as follows:

[0094]

[0095] Among them, n and m are the number of queries and key-value pairs respectively. The attention mechanism module can adopt a multi-head attention mechanism or a single-head attention mechanism. The multi-head attention module allows the model to use a single attention head to jointly focus on information at different positions from subspaces of different representations. For a single attention head, this feature is suppressed on average. The formula of the multi-head attention mechanism is denoted as MultiHead(Q,K,V). The function representation of the multi-head attention mechanism accepts three inputs: query (Query, denoted as Q), key (Key, denoted as K) and value (Value, denoted as V). As shown below:

[0096] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0097] head i =Attention(QW i Q ,KW i K ,VW i V )

[0098] Among them, head1,…,head h Represents the transformation matrices of h single-head attention modules, i-th query, key, and value respectively Among them, the final output transformation matrix is Concat represents a concatenation operation. It is preferred to use h = 8 parallel attention layers, d model Refers to the dimension of the vector in the model, where d k =d v =d model / h=64, specifies that 8 attention heads are used, and the key (d k ) and value (d v ) is 64. Due to the reduced size of each head, the total computational cost of the multi-head attention mechanism is similar to that of the single-head attention mechanism with full dimensions. i denotes the identifier of different network layers.

[0099] The feedforward neural network module FFN can be set as a multi-layer perceptron network (MLP, Multi-LayerPerceptron), which can be considered as an implementation of a fully connected feedforward network. It consists of two linear change layers, and there is a ReLU activation function between the two linear transformation layers:

[0100] FFN(x)=ReLU(xW1+b1)W2+b2

[0101] in, After that, residual connection and layer normalization are performed. The adaptive network (Adapter structure) is connected to the attention mechanism module and the feed-forward neural network module of each Transformer unit, and the adaptive network is trained during the process. x represents the output of a certain layer.

[0102] There are various ways for the adaptive network to be connected to the Transformer block. For example, it can be connected after the multi-head attention mechanism module or the feed-forward neural network module, or in parallel with both, or even inserted after the residual layer (denoted as the Add&Norm layer). The predetermined replacement strategy can also include different positions for the connection of the adaptive network, such as parallel setting (parallel adapter) and sequential setting (sequence adpater).

[0103] The architecture of the adaptive network can be a bottleneck architecture, which is a neural network structure. First, the original d-dimensional input representation is projected to a low dimension r (r << d) through a lower projection matrix , then a non-linear activation function g(.) and a sub-learning module f(.) are used to learn the hidden features in the input data, and then the r-dimensional representation is projected back to the d dimension through an upper projection matrix . Let r be the dimension of the bottleneck, and let be the output of each adapter module (adaptive network). The formula for each adapter module is expressed as:

[0104]

[0105] Among them, the non-linear activation function g(.) includes but is not limited to RELU, GELU, and Tanh, or the identity transformation. The above three different non-linear activation functions are compared, and the best-performing non-linear activation function is selected for the training of the model and the subsequent inference process. The equations of the activation functions RELU, GELU, and Tanh are expressed as follows:

[0106] g(x) = max(0, x)

[0107]

[0108] To improve the classification efficiency of the target model, four different settings are added to the adapter module (i.e., the adaptive network) to distinguish different sub-learning modules f(.) to learn the hidden features of the samples, so as to enhance the model's ability to extract information from the hidden layer. These mainly include four ways of setting the adaptive network as PF-BERxiT(origin-adapter), PF-BERxiT(self-attn-adapter), PF-BERxiT(conv-adapter), and PF-BERxiT(mlp-adapter).

[0109] When the adaptive network is set as PF-BERxiT(origin-adapter), that is, it only contains the lower projection matrix The original d-dimensional input representation is projected to a low dimension r (r << d), then connected to the non-linear activation function g(.), and then projected back to d dimensions through the upper projection matrix That is, there is no sub-learning module in the adapter structure (adaptive network). Under this condition, the formula of the adapter is:

[0110]

[0111] When the adaptive network is set as PF-BERxiT(self-attn-adapter), the sub-learning module is the self-attention mechanism layer, that is, it contains the lower projection matrix The original d-dimensional input representation is projected to a low dimension r (r << d), then connected to the non-linear activation function g(.) and the sub-learning module self-attention mechanism layer, and then a non-linear activation function can be selectively added. This activation function can be RELU, GELU, and Tanh, or the identity transformation. Finally, it is projected back to d dimensions through the upper projection matrix That is, there is a single-layer attention mechanism sub-learning module in the adapter structure. Under this condition, the formula of the adapter is:

[0112]

[0113] When the adaptive network is set as PF-BERxiT(self-attn-adapter), the sub-learning module is a one-dimensional convolutional layer, that is, it contains the lower projection matrix The original d-dimensional input representation is projected to a low dimension r (r << d), then connected to the non-linear activation function g(.) and the sub-learning module one-dimensional convolutional layer, and then a non-linear activation function can be selectively added. This activation function can be RELU, GELU, and Tanh, or the identity transformation. Finally, it is projected back to d dimensions through the upper projection matrix Project the r-dimensional representation back to the d-dimension. That is, there is a one-dimensional convolutional neural network sub-learning module in the adapter structure. Under this condition, the formula of the adapter is as follows:

[0114]

[0115] The adaptive network is set to PF-BERxiT(mlp-adapter), and the sub-learning module is a simple fully connected layer, that is, it contains a lower projection matrix Project the original d-dimensional input representation to a low dimension r (r << d), then connect a non-linear activation function g(.) and a simple fully connected layer of the sub-learning module, and then a non-linear activation function can be selectively added. This activation function can be RELU, GELU, and Tanh, or an identity transformation. Finally, project the r-dimensional representation back to the d-dimension through the upper projection matrix Project the r-dimensional representation back to the d-dimension. That is, there is a fully connected sub-learning module in the adapter structure. Under this condition, the formula of the adapter is as follows:

[0116]

[0117] At least any one of the following effects can be achieved by the above optional implementation manners: A new type of parameter-efficient transfer learning model integrating a sub-learning module is proposed, that is, a model containing a sub-learning module is connected to each Transformer layer, which alleviates the problems of high computational cost and low efficiency in the training process. The sub-learning module is a bottleneck structure, which can map high-dimensional parameters to low dimensions, and a Conv / MLP / Attention module is added to improve the model's ability to learn deep association characteristics. Under similar training parameters, higher accuracy can be obtained. Only the parameters of the adapter module are trained during training. This method can train fewer parameters to achieve an effect equivalent to training all the parameters of the model, shortening the training time and saving computational resources.

[0118] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0119] In this embodiment, a text classification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0120] According to an embodiment of the present invention, a device embodiment for implementing the text classification method is also provided. Figure 3 is a schematic diagram of a text classification device according to an embodiment of the present invention. Figure 3 As shown, the above-mentioned text classification device includes: an input module 302, a classification module 304. The device is described below.

[0121] An input module 302 is used to process the multiple network layers included in the input target model based on the input text vector to determine the semantic dependency features, wherein the multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is less than the parameter amount of the corresponding network layer, and the target model completes the model training process by adjusting the parameters included in the adaptive network;

[0122] The classification module 304 is connected to the input module 302 and is used to process the semantic dependency features using the classifier included in the target model to obtain a target classification result of the input text vector.

[0123] In a text classification device provided by an embodiment of the present invention, an input module 302 is used to process multiple network layers included in an input target model based on an input text vector to determine semantic dependency features, wherein multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is less than the parameter amount of the corresponding network layer, and the target model completes model training processing by adjusting the parameters included in the adaptive network; a classification module 304 is connected to the input module 302 and is used to process the semantic dependency features using a classifier included in the target model to obtain a target classification result of the input text vector. The purpose of improving the accuracy of text classification under limited training resources is achieved, the amount of parameters required for model training is reduced, and the technical effect of maintaining the accuracy of text classification is achieved, thereby solving the technical problem of unsatisfactory accuracy of text classification under the condition of limited training resources in the related art.

[0124] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0125] It should be noted that the input module 302 and the classification module 304 correspond to steps S102 to S104 in the embodiment, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in the embodiment. It should be noted that the modules as part of the device can be run in a computer terminal.

[0126] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0127] The above-mentioned text classification device may also include a processor and a memory. The input module 302, the classification module 304, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0128] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0129] An embodiment of the present invention provides a non-volatile storage medium on which a program is stored. When the program is executed by a processor, a text classification method is implemented.

[0130] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: based on an input text vector, multiple network layers included in an input target model are processed to determine semantic dependency features, wherein the multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is less than the parameter amount of the corresponding network layer, and the target model is a model training process completed by adjusting the parameters included in the adaptive network; the semantic dependency features are processed by a classifier included in the target model to obtain a target classification result of the input text vector. The device in this article can be a server, a PC, etc.

[0131] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: based on an input text vector, multiple network layers included in an input target model are processed to determine semantic dependency features, wherein the multiple network layers are respectively connected to an adaptive network, the parameter amount of the adaptive network is smaller than the parameter amount of the corresponding network layer, and the target model completes model training processing by adjusting the parameters included in the adaptive network; the classifier included in the target model is used to process the semantic dependency features to obtain a target classification result of the input text vector.

[0132] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0140] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0141] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A text classification method, characterized in that: include: Based on the input text vector, the input target model includes multiple network layers for processing to determine the semantic dependency features, wherein the multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is smaller than the parameter amount of the corresponding network layer, and the target model completes the model training process by adjusting the parameters included in the adaptive network; The semantic dependency features are processed using a classifier included in the target model to obtain a target classification result of the input text vector.

2. The method according to claim 1, characterized in that The multiple network layers further include corresponding attention mechanism modules and feedforward neural network modules respectively, and the attention mechanism modules and the feedforward neural network modules are connected via a residual module, wherein the attention mechanism module is used to extract features at different positions in the input text vector, and the feedforward neural network module is used to increase the nonlinear ability of the corresponding network layer; According to a predetermined replacement strategy, the adaptive network is connected to the corresponding network layer by replacing the attention mechanism module and / or the feedforward neural network module included in the corresponding network layer, wherein the predetermined replacement strategies are respectively set for the multiple network layers.

3. The method according to claim 1, characterized in that The multiple network layers are stacked in a predetermined order, and the multiple network layers included in the input target model are processed based on the input text vector to determine the semantic dependency features, including: Inputting the input text vectors into the multiple network layers in sequence for processing according to the predetermined order; The output features of the previous network layer included in the multiple network layers are used as the input features of the next network layer, and the output features of the last network layer included in the multiple network layers are obtained as the semantic dependency features, wherein the multiple network layers are used to extract text features at different levels respectively.

4. The method according to claim 3, characterized in that The step of using the output features of a previous network layer included in the multiple network layers as the input features of a subsequent network layer includes: Performing dimensionality reduction processing based on the output features of the previous network layer to obtain dimensionality reduction features; Performing nonlinear processing based on the dimension reduction feature to obtain a nonlinear feature; In the case where the adaptive network includes a sub-learning module, the sub-learning module is used to perform feature extraction based on the nonlinear feature to obtain a target feature, wherein the sub-learning module is used to enhance the ability of extracting features; Dimensionality up-scaling is performed based on the target feature to obtain the input feature of the subsequent network layer.

5. The method according to claim 4, characterized in that The sub-learning module is set to at least any one of the following: a self-attention layer, a convolutional layer, and a fully connected layer.

6. The method according to claim 4, characterized in that The method further comprises: In the case that the adaptive network does not include a sub-learning module, a dimensionality increase process is performed based on the nonlinear features to obtain input features of the latter network.

7. The method according to any one of claims 1 to 6, characterized in that: The multiple network layers are stacked in a predetermined order, and the multiple network layers are respectively provided with sub-classifiers. The method further includes: Using the sub-classifiers respectively set for the multiple network layers, classify the output features respectively generated by the multiple network layers, and obtain sub-classification results respectively corresponding to the multiple network layers; Determine consistency results between sub-classification results corresponding to a predetermined number of network layers in the plurality of network layers from front to back according to the predetermined order indication; If the consistency result satisfies a predetermined condition, stopping processing of a network layer after the predetermined number of network layers among the multiple network layers; The target classification result is determined based on the sub-classification results respectively corresponding to the predetermined number of network layers.

8. A text classification device, characterized in that: include: An input module, used for processing multiple network layers included in the input target model based on the input text vector, and determining semantic dependency features, wherein the multiple network layers are respectively connected to adaptive networks, the parameter amount of the adaptive network is smaller than the parameter amount of the corresponding network layer, and the target model completes the model training process by adjusting the parameters included in the adaptive network; The classification module is used to process the semantic dependency features using the classifier included in the target model to obtain a target classification result of the input text vector.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the text classification method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the text classification method described in any one of claims 1 to 7.

Citation Information

Cited By

  • Archive file information processing method, device and equipment

    CN119311894A