Text multi-label classification method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and finance, and relates to a text multi-label classification method, which comprises the following steps: preprocessing an original text data set, obtaining classification labels of the preprocessed text data set, marking the classification labels, obtaining a marked text data set, and dividing the marked text data set into a training set and a verification set; constructing a low-rank adaptive classification model according to the classification label, inputting the training set into the low-rank adaptive classification model, and obtaining a classification prediction result fusing a semantic feature extraction result and a low-rank decomposition result; fine-tuning the low-rank adaptive classification model based on the classification prediction result, and continuing to train to obtain a fine-tuning classification model; verifying the fine-tuning classification model to obtain a final text label classification model; and inputting the to-be-classified text into the text label classification model for classification. The invention further provides a text multi-label classification device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the tagged text data set can be stored in a block chain. The text multi-label classification accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a text multi-label classification method, apparatus, computer equipment and storage medium. Background Art

[0002] With the rapid development of the Internet, a massive amount of text data has been generated in network scenarios. This data covers multiple fields such as news, social media, academic papers, business reports, etc. For enterprises and research institutions, efficiently understanding and utilizing this information has become the key to improving competitiveness. In order to achieve this goal, it is particularly important to accurately classify and categorize text data. Traditional text classification methods, such as rule-based methods or statistical machine learning-based methods, usually rely on manually extracted features or the knowledge of domain experts. These methods are not only time-consuming and labor-intensive, but their classification effects are often limited to specific fields or data sets, resulting in limited applicability and scalability.

[0003] In recent years, the rapid development of artificial intelligence, especially the breakthrough of natural language processing (NLP) technology, has provided new solutions for text classification. Super large language models, such as GPT3, LLaMA and ChattGLM, have made remarkable achievements in the field of natural language processing with their powerful language generation and understanding capabilities. These models are able to learn rich language knowledge and contextual information through pre-training on large-scale corpora, thus showing excellent performance in various semantic understanding and generation tasks.

[0004] However, despite the huge success of very large language models in the field of NLP, their application still faces challenges in the scenario of multi-label text classification. Multi-label text classification means that a text may correspond to multiple labels at the same time. This complexity makes the classification task more difficult. Traditional text classification methods based on very large language models often adopt the method of independently predicting each label, ignoring the possible correlation between different labels. This isolated processing method limits the model's ability to capture complex semantic information in the text. Especially in the financial field, when faced with a large amount of text data involving complex information such as market dynamics, economic policies, and corporate financial reports, it is more difficult to capture the correlation between different labels, resulting in deviations or omissions in the classification results, affecting the overall performance of the classification effect. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a text multi-label classification method, apparatus, computer equipment and storage medium to solve the technical problem of poor accuracy and stability of text multi-label classification in related technologies.

[0006] In order to solve the above technical problems, the present application embodiment provides a text multi-label classification method, which adopts the following technical solution:

[0007] Acquire an original text data set, and preprocess the original text data set to obtain a preprocessed text data set;

[0008] Obtaining a classification label of each preprocessed text in the preprocessed text dataset, marking the preprocessed text dataset according to the classification label to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a validation set;

[0009] Constructing a low-rank adaptive classification model according to the classification labels, and inputting the training set into the low-rank adaptive classification model;

[0010] Obtaining a semantic feature extraction result and a low-rank decomposition result of the training set through the low-rank adaptive classification model, and fusing the semantic feature extraction result and the low-rank decomposition result to obtain a classification prediction result;

[0011] Fine-tuning the low-rank adaptive classification model based on the classification prediction result, continuing iterative training until an iteration stop condition is met, and obtaining a fine-tuned classification model;

[0012] Inputting the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, outputting the fine-tuning classification model as a final text label classification model;

[0013] The text to be classified is obtained, and the text to be classified is input into the text label classification model for classification to obtain a text classification result.

[0014] Furthermore, the step of obtaining the classification label of each preprocessed text in the preprocessed text dataset includes:

[0015] The TF-IDF algorithm is used to extract keyword features of each preprocessed text in the preprocessed text dataset;

[0016] Using a clustering algorithm, classifying the preprocessed text according to the keyword features to obtain different text categories;

[0017] A classification label for each of the preprocessed texts is generated according to the keyword features of the cluster centers of the text categories.

[0018] Furthermore, the low-rank adaptive classification model includes an input layer, a pre-trained super-large language model, a low-rank adaptive network and multiple classifiers, and the steps of obtaining the semantic feature extraction result and the low-rank decomposition result of the training set through the low-rank adaptive classification model, and fusing the semantic feature extraction result and the low-rank decomposition result to obtain the classification prediction result include:

[0019] Preprocessing the training set through the input layer to obtain input text data;

[0020] Inputting the input text data into the pre-trained super large language model to extract semantic features, and obtaining a text semantic representation vector;

[0021] Performing low-rank decomposition on the input text data through the low-rank adaptive network to obtain a text low-rank adaptive matrix;

[0022] Concatenating the text semantic representation vector and the text low-rank adaptation matrix to obtain text semantic features;

[0023] The text semantic features are respectively input into the multiple classifiers for classification to obtain classification prediction results.

[0024] Furthermore, the super large language model includes an embedding layer and a multi-level feature extraction network, and the step of inputting the input text data into the pre-trained super large language model for semantic feature extraction to obtain a text semantic representation vector includes:

[0025] Performing vector conversion on the input text data through the embedding layer to obtain a text embedding vector;

[0026] The text embedding vector is input into the multi-level feature extraction network to perform feature extraction at different scales, so as to obtain a multi-scale fused text semantic representation vector.

[0027] Furthermore, the low-rank adaptive network includes a vector conversion layer and a low-rank adaptive matrix corresponding to each of the classification labels; the step of performing low-rank decomposition on the input text data through the low-rank adaptive network to obtain a text low-rank adaptive matrix includes:

[0028] Performing vector conversion on the text in the input text data through the vector conversion layer to obtain a text word vector;

[0029] Input the text word vectors into each of the low-rank adaptation matrices for matrix decomposition to obtain a text feature matrix corresponding to each of the classification labels;

[0030] All the text feature matrices are concatenated to obtain a text low-rank adaptation matrix.

[0031] Furthermore, the step of inputting the text semantic features into the multiple classifiers for classification to obtain classification prediction results includes:

[0032] Inputting the text semantic features into the multiple classifiers respectively, wherein the multiple classifiers include classifiers corresponding to the respective classification labels;

[0033] Performing classification prediction on the text semantic features through each of the classifiers to obtain a prediction probability of each of the classification labels;

[0034] The classification labels whose prediction probabilities are greater than or equal to a preset confidence threshold are output as classification prediction results.

[0035] Furthermore, the step of fine-tuning the low-rank adaptive classification model based on the classification prediction result includes:

[0036] Calculate the loss value between the classification prediction result and the classification label according to a preset loss function;

[0037] Based on the loss value, a back propagation algorithm is used to adjust the parameters of the low-rank adaptive network and the classifier.

[0038] In order to solve the above technical problems, the embodiment of the present application also provides a text multi-label classification device, which adopts the following technical solution:

[0039] A preprocessing module, used to obtain an original text data set, preprocess the original text data set, and obtain a preprocessed text data set;

[0040] A labeling module, used for obtaining a classification label of each preprocessed text in the preprocessed text data set, labeling the preprocessed text data set according to the classification label to obtain a labeled text data set, and randomly dividing the labeled text data set into a training set and a validation set;

[0041] A construction module, used to construct a low-rank adaptive classification model according to the classification label, and input the training set into the low-rank adaptive classification model;

[0042] A prediction module, used to obtain a semantic feature extraction result and a low-rank decomposition result of the training set through the low-rank adaptive classification model, and to fuse the semantic feature extraction result and the low-rank decomposition result to obtain a classification prediction result;

[0043] An iterative module, used for fine-tuning the low-rank adaptive classification model based on the classification prediction result, continuing iterative training until an iteration stop condition is met, and obtaining a fine-tuned classification model;

[0044] A verification module, used for inputting the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, outputting the fine-tuning classification model as a final text label classification model;

[0045] The classification module is used to obtain the text to be classified, input the text to be classified into the text label classification model for classification, and obtain the text classification result.

[0046] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0047] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned text multi-label classification method when executing the computer-readable instructions.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0049] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the text multi-label classification method described above are implemented.

[0050] Compared with the prior art, this application has the following beneficial effects:

[0051] The present application provides a text multi-label classification method, which can improve data quality and enhance model generalization ability by preprocessing the original text data set; by obtaining the classification label of the preprocessed text and marking it, it can enhance the expression ability of the classification feature, which helps to capture the correlation between different classification labels to achieve accurate multi-label classification; by inputting the training set into a low-rank adaptive classification model for processing, the semantic feature extraction result and low-rank decomposition result of the training set are obtained, and the classification prediction result is obtained by fusing the semantic feature extraction result and the low-rank decomposition result, which can make full use of the semantic representation ability of the model, capture the deep semantic information of the text, and use fewer parameters to identify the key information of the text through low-rank decomposition, thereby capturing the correlation between different labels, improving the parameter efficiency of the model while improving the accuracy and stability of text multi-label classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0054] Figure 2 is a flowchart of an embodiment of a text multi-label classification method according to the present application;

[0055] Figure 3 is a structural schematic diagram of an embodiment of a text multi-label classification device according to the present application;

[0056] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0058] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0060] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0061] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0062] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0063] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0064] It should be noted that the text multi-label classification method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the text multi-label classification device is generally set in the server / terminal device.

[0065] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0066] Continue to refer Figure 2 , shows a flowchart of an embodiment of a text multi-label classification method according to the present application, comprising the following steps:

[0067] Step S201, obtaining an original text data set, preprocessing the original text data set, and obtaining a preprocessed text data set.

[0068] Initiate data acquisition requests based on business needs, obtain a large amount of business data in the corresponding business scenarios based on the data acquisition requests, and form original text data sets. The sources of original text data sets include but are not limited to financial databases and third-party research institutions, internal data of financial institutions, financial media and news websites, and regulatory agencies.

[0069] In this embodiment, the text multi-label classification method is executed on the electronic device (eg Figure 1The server / terminal device shown in the figure can initiate a data acquisition request through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0070] The obtained original text dataset is preprocessed. The preprocessing operations include cleaning, normalization, word segmentation, etc. Cleaning includes removing irrelevant characters and special symbols, removing stop words, spelling checking, grammar correction, removing duplicate content, filling or deleting missing values, etc. Normalization is to use the normalization method to scale the data values ​​to the range of [0,1]. Specifically, the original text data in the original text dataset is cleaned to obtain a cleaned text dataset; the cleaned text dataset is normalized to obtain a normalized text dataset; the normalized text dataset is segmented by word segmentation technology to obtain a preprocessed text dataset containing word segmentation units.

[0071] Step S202, obtaining the classification label of each preprocessed text in the preprocessed text dataset, marking the preprocessed text dataset according to the classification label to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a validation set.

[0072] In this embodiment, the preprocessed text data set is classified to obtain the corresponding classification label for each preprocessed text in the preprocessed text data set, and professional text annotation tools such as LabelImg, Labelme, etc. are used to annotate the text data in the preprocessed text data set with classification labels to obtain an annotated text data set containing text data and classification labels.

[0073] In some optional implementations, the step of obtaining a classification label for each preprocessed text in the preprocessed text dataset includes:

[0074] The TF-IDF algorithm is used to extract the keyword features of each preprocessed text in the preprocessed text dataset;

[0075] Using clustering algorithms, the preprocessed text is classified according to keyword features to obtain different text categories;

[0076] According to the keyword features of the cluster center of each text category, a classification label for each preprocessed text is generated.

[0077] Among them, the TF-IDF algorithm is used to calculate the ratio of the frequency of each word segmentation unit in the text in which it is located and the frequency of occurrence in all texts, that is, the importance weight; the word segmentation units with importance weights greater than or equal to the preset weight threshold are screened out as the keyword features of the corresponding preprocessed text. Through the K-means clustering algorithm, K initial clustering centers are determined according to the keyword features, where K is a positive integer; all keyword features are traversed, the distance between the keyword features and each initial clustering center is calculated, and the group corresponding to the keyword features is determined; the cluster center of each group is updated to obtain the actual cluster center of each group; the distance between the keyword features and each actual cluster center is repeatedly calculated, and the cluster center of each group is continuously updated until the group no longer changes, and different keyword categories are obtained; the similarity scores of all keyword features of each preprocessed text and each keyword category are calculated, and the keyword category with a similarity score greater than or equal to the preset similarity threshold is used as the text classification category to which the preprocessed text belongs, and the classification label corresponding to the text classification category is automatically generated according to the keyword features corresponding to the cluster center of each text classification category, that is, the classification label of each preprocessed text is generated, and one preprocessed text may correspond to multiple classification labels.

[0078] By obtaining the classification labels of the preprocessed text dataset, the expressive power of the classification features can be enhanced, so that the model can better capture the key information of different classification labels, so as to improve the generalization ability and training speed of the model, thereby improving the accuracy, robustness and efficiency of text classification.

[0079] In this embodiment, the annotated text data set is randomly divided into a training set and a validation set according to a preset ratio. The preset ratio can be set according to actual needs. For example, the preset ratio is 8:2, that is, the ratio of the training set to the validation set is 8:2.

[0080] It should be emphasized that in order to further ensure the privacy and security of the annotated text dataset, the above-mentioned annotated text dataset can also be stored in a node of a blockchain.

[0081] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0082] Step S203, construct a low-rank adaptive classification model according to the classification labels, and input the training set into the low-rank adaptive classification model.

[0083] In this embodiment, all classification labels are obtained, and a low-rank adaptive matrix corresponding to each classification label is constructed through feature extraction and dimensionality reduction algorithms. All low-rank adaptive matrices are integrated to obtain a low-rank adaptive network. According to the classification label, a classifier corresponding to each classification label is constructed. A pre-trained super large language model is obtained, an input layer is constructed, and the input layer, the pre-trained super large language model, the low-rank adaptive network and all classifiers are integrated to obtain a low-rank adaptive classification model. The low-rank adaptive classification model is trained through a training set.

[0084] In order to better adapt to the text multi-label classification task, a low-rank adaptation network is introduced for fine-tuning. A low-rank adaptation matrix is ​​constructed for each classification label. The low-rank adaptation matrix includes two low-rank matrices B and A, B∈R M×N , A∈R N ×M , where N is the rank of the two low-rank matrices, and the rank N is extremely smaller than M.

[0085] In this embodiment, a low-rank adaptive network is introduced, so that the model can capture the correlation between multiple labels while reducing the number of parameters, supporting subsequent rapid fine-tuning to generate a text classification model.

[0086] Step S204, obtaining the semantic feature extraction result and the low-rank decomposition result of the training set through the low-rank adaptive classification model, and fusing the semantic feature extraction result and the low-rank decomposition result to obtain the classification prediction result.

[0087] The low-rank adaptive classification model includes an input layer, a pre-trained very large language model, a low-rank adaptive network and multiple classifiers. The training set is input into the low-rank adaptive classification model, and the training set is preprocessed through the input layer to obtain input text in a data format suitable for model processing. The input text is input into the pre-trained very large language model and the low-rank adaptive network respectively, and the results obtained from the two are added together to obtain the final semantic result. The final semantic result is then input into classifiers with different classification labels for classification training to obtain classification prediction results.

[0088] Specifically, the training set is preprocessed through the input layer to obtain input text data; the input text data is input into the pre-trained very large language model to extract semantic features and obtain a text semantic representation vector; the input text data is low-rank decomposed through a low-rank adaptive network to obtain a text low-rank adaptive matrix; the text semantic representation vector and the text low-rank adaptive matrix are concatenated to obtain text semantic features; the text semantic features are respectively input into multiple classifiers for classification to obtain classification prediction results.

[0089] In this embodiment, by training a low-rank adaptive classification model for text classification, the advantages of the super-large language model in semantic understanding can be fully utilized, the information in the text data can be better captured, and the key features of different classification labels can be captured through the low-rank adaptive network to achieve the distinction and capture of the correlation between different labels, thereby improving the accuracy and stability of multi-label classification.

[0090] In some optional implementations, the super large language model includes an embedding layer and a multi-level feature extraction network, and the step of inputting the input text data into the pre-trained super large language model for semantic feature extraction to obtain the text semantic representation vector includes:

[0091] The input text data is converted into a vector through the embedding layer to obtain a text embedding vector;

[0092] The text embedding vector is input into a multi-level feature extraction network to extract features at different scales and obtain a multi-scale fused text semantic representation vector.

[0093] In this embodiment, the vector conversion of the embedding layer includes position embedding, word embedding and character embedding of the input text data, wherein position embedding provides information about the position of the word in the sentence, which helps to understand the grammatical structure and contextual relationship of the sentence; word embedding captures the semantic information of the vocabulary, so that the model can understand the meaning of the vocabulary in the context; character embedding captures the information at the character level, which helps the model to process unknown vocabulary or spelling variants, and improves the model's ability to process complex data by incorporating character-level information into the model. The text embedding vector is a concatenated vector of the position embedding vector, the word embedding vector and the character embedding vector.

[0094] The multi-level feature extraction network can use encoder structure, encoder-decoder structure and decoder structure to extract features of different scales.

[0095] When the multi-level feature extraction network adopts an encoder structure, the encoder structure is formed by stacking multiple layers of encoders, and the multiple layers of encoders are divided into bottom-level feature units, middle-level feature units and high-level feature units. The surface features of the text embedding vector are extracted through the bottom-level feature units, and the extracted surface features are input into the middle-level feature units and the high-level feature units respectively; the surface features are extracted through the middle-level feature units to obtain syntactic features, and the syntactic features are input into the high-level feature units; the surface features and syntactic features are extracted through the high-level feature units to obtain semantic features; the surface features, syntactic features and semantic features are concatenated to obtain a text semantic representation vector.

[0096] When the multi-level feature extraction network adopts an encoder-decoder structure, the encoder-decoder structure includes multiple stacked encoders and multiple stacked decoders. The encoder multi-head attention mechanism extracts the semantic and contextual features of the text embedding vector to obtain the text encoding features, and the text encoding features are input into the decoder. The global dependency between the text encoding features is captured through the multi-head attention mechanism and the cross attention mechanism to obtain the text semantic representation vector. Among them, the encoder and the decoder process text features of different scales through the multi-head attention mechanism.

[0097] By extracting semantic features through a super-large language model, we can better capture the deep semantic information of the text and identify the classification features in the text data, which in turn helps to effectively capture label relevance and improve the efficiency and accuracy of text classification.

[0098] In some optional implementations, the low-rank adaptive network includes a vector conversion layer and a low-rank adaptive matrix corresponding to each of the classification labels, and the step of performing low-rank decomposition on the input text data through the low-rank adaptive network to obtain the text low-rank adaptive matrix includes:

[0099] The text in the input text data is converted into a vector through the vector conversion layer to obtain the text word vector;

[0100] The text word vectors are input into each low-rank adaptation matrix for matrix decomposition to obtain the text feature matrix corresponding to each classification label;

[0101] Concatenate all text feature matrices to obtain the text low-rank adaptation matrix.

[0102] In this embodiment, the input text data is converted into text word vectors, and the text word vectors are respectively input into each low-rank adaptation matrix. The text word vectors are matrix decomposed by singular value decomposition (SVD) to capture the key features of each classification label and form a text low-rank adaptation matrix.

[0103] Low-rank decomposition through a low-rank adaptive network can identify the differences and associations between different classification labels. At the same time, it reduces the number of parameters that need to be adjusted, thereby improving model training efficiency and classification effects.

[0104] The text low-rank adaptation matrix and the text semantic representation vector are concatenated. The concatenation method can be to add or multiply the text low-rank adaptation matrix and the text semantic representation vector to obtain the final text semantic features, and then input the text semantic features into classifiers of different classification labels for classification prediction.

[0105] In this embodiment, the step of inputting the text semantic features into a plurality of classifiers for classification to obtain classification prediction results includes:

[0106] Inputting the text semantic features into a plurality of classifiers respectively, wherein the plurality of classifiers include classifiers corresponding to the respective classification labels;

[0107] Use each classifier to classify and predict the semantic features of the text and obtain the predicted probability of each classification label;

[0108] The classification labels whose prediction probabilities are greater than or equal to the preset confidence threshold are output as classification prediction results.

[0109] In this embodiment, the classifier can use a fully connected layer to map the text semantic features to the classification label corresponding to the classifier through the Softmax activation function, obtain the classification probability distribution of each classification label, and obtain the predicted probability of each classification label based on the classification probability distribution.

[0110] The prediction probability is usually expressed as prediction confidence. For example, if the classifier predicts that the probability of the corresponding classification label is 0.9, then 0.9 can be regarded as the confidence of the classifier in the prediction result.

[0111] In some optional implementations of this embodiment, the classifier may adopt a support vector machine (SVM). SVM separates texts with different classification labels by finding the optimal decision hyperplane, inputs the text semantic features into the SVM classifier, and determines the classification label to which the text belongs through the position of the decision hyperplane, thereby completing the text classification task.

[0112] By training the classifier, the text multi-label classification task can be completed more quickly and the processing efficiency can be improved. At the same time, the accuracy of text classification can be improved while reducing misjudgments and missed judgments.

[0113] Step S205, fine-tune the low-rank adaptive classification model based on the classification prediction result, continue iterative training until the iteration stop condition is met, and obtain a fine-tuned classification model.

[0114] Specifically, the loss value between the classification prediction result and the classification label is calculated according to the preset loss function; based on the loss value, the back propagation algorithm is used to adjust the parameters of the low-rank adaptive network and the classifier.

[0115] In this embodiment, the preset loss function can select the cross entropy loss function, which can measure the difference between the model prediction result and the true label. The loss value between the classification prediction result and the classification label is calculated, and the gradient of the loss value to the model parameter is returned through the back propagation algorithm to adjust the parameters of the low-rank adaptive network and the classifier to achieve the model fine-tuning process. Continue to iterate the fine-tuned model until the iteration stop condition is met to obtain a converged fine-tuned classification model. Among them, the iteration stop condition is to reach the preset maximum number of iterations, or the current loss value no longer decreases compared to the loss value of the previous round.

[0116] In some optional implementations of this embodiment, the overall recognition range can be well modified by adding or deleting low-rank matrices.

[0117] Since the subsequent training and fine-tuning of the ultra-large language model requires a lot of computing resources, this embodiment chooses to freeze its parameters and only adjusts the parameters of the low-rank adaptive network and the classifier. Through fine-tuning, the model can better adapt to multi-label classification tasks, reduce training time and computing resources, reduce dependence on large amounts of labeled data, and enhance the model's generalization ability.

[0118] Step S206, inputting the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, outputting the fine-tuning classification model as the final text label classification model.

[0119] The verification set is verified by fine-tuning the classification model to obtain the verification classification result, and the classification accuracy between the verification classification result and the corresponding true classification label is calculated. The classification accuracy is used as the verification result. When the classification accuracy is greater than or equal to the preset threshold, it means that the verification result meets the preset threshold condition, and the current fine-tuned classification model is output as the final text label classification model; when the classification accuracy is less than the preset threshold, it means that the verification result does not meet the preset threshold condition, then return to step S204 and retrain the low-rank adaptive classification model.

[0120] Step S207, obtaining the text to be classified, inputting the text to be classified into the text label classification model for classification, and obtaining a text classification result.

[0121] The trained text label classification model is used to perform multi-label text classification on the text to be classified to obtain a text classification result, which includes the classification label to which the text to be classified belongs.

[0122] In this embodiment, the text label classification model is regularly adjusted and optimized to obtain a text label classification model with stronger generalization ability and robustness to adapt to complex and changing financial business scenarios.

[0123] By preprocessing the original text dataset, the present application can improve data quality and enhance the generalization ability of the model; by obtaining the classification labels of the preprocessed text and marking them, the expressiveness of the classification features can be enhanced, which helps to capture the correlation between different classification labels to achieve accurate multi-label classification; by inputting the training set into a low-rank adaptive classification model for processing, the semantic feature extraction results and low-rank decomposition results of the training set are obtained, and the classification prediction results are obtained by fusing the semantic feature extraction results and the low-rank decomposition results, which can make full use of the semantic representation ability of the model, capture the deep semantic information of the text, and use fewer parameters through low-rank decomposition to identify the key information of the text, thereby capturing the correlation between different labels, improving the parameter efficiency of the model while improving the accuracy and stability of text multi-label classification.

[0124] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0125] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0127] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0128] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a text multi-label classification device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0129] like Figure 3 As shown, the text multi-label classification device 300 described in this embodiment includes: a preprocessing module 301, a labeling module 302, a construction module 303, a prediction module 304, an iteration module 305, a verification module 306 and a classification module 307. Among them:

[0130] The preprocessing module 301 is used to obtain an original text data set, preprocess the original text data set, and obtain a preprocessed text data set;

[0131] The labeling module 302 is used to obtain the classification label of each preprocessed text in the preprocessed text data set, label the preprocessed text data set according to the classification label to obtain a labeled text data set, and randomly divide the labeled text data set into a training set and a validation set;

[0132] The construction module 303 is used to construct a low-rank adaptive classification model according to the classification label, and input the training set into the low-rank adaptive classification model;

[0133] The prediction module 304 is used to obtain the semantic feature extraction result and the low-rank decomposition result of the training set through the low-rank adaptive classification model, and fuse the semantic feature extraction result and the low-rank decomposition result to obtain the classification prediction result;

[0134] The iteration module 305 is used to fine-tune the low-rank adaptive classification model based on the classification prediction result, and continue iterative training until the iteration stop condition is met to obtain a fine-tuned classification model;

[0135] The verification module 306 is used to input the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, output the fine-tuning classification model as the final text label classification model;

[0136] The classification module 307 is used to obtain the text to be classified, input the text to be classified into the text label classification model for classification, and obtain a text classification result.

[0137] It should be emphasized that in order to further ensure the privacy and security of the annotated text dataset, the above-mentioned annotated text dataset can also be stored in a node of a blockchain.

[0138] Based on the above-mentioned text multi-label classification device 300, by preprocessing the original text data set, the data quality can be improved and the model generalization ability can be enhanced; by obtaining the classification label of the preprocessed text and marking it, the expression ability of the classification feature can be enhanced, which helps to capture the correlation between different classification labels to achieve accurate multi-label classification; by inputting the training set into the low-rank adaptive classification model, fusing the semantic feature extraction result and the low-rank decomposition result to obtain the classification prediction result, the semantic representation ability of the model can be fully utilized to capture the deep semantic information of the text, and the key information of the text can be identified by using fewer parameters through low-rank decomposition, thereby capturing the correlation between different labels, improving the parameter efficiency of the model while improving the accuracy and stability of text multi-label classification.

[0139] In some optional implementations of this embodiment, the marking module 302 is further used to:

[0140] The TF-IDF algorithm is used to extract keyword features of each preprocessed text in the preprocessed text dataset;

[0141] Using a clustering algorithm, classifying the preprocessed text according to the keyword features to obtain different text categories;

[0142] A classification label for each of the preprocessed texts is generated according to the keyword features of the cluster centers of the text categories.

[0143] By obtaining the classification labels of the preprocessed text dataset, the expressive power of the classification features can be enhanced, so that the model can better capture the key information of different classification labels, so as to improve the generalization ability and training speed of the model, thereby improving the accuracy, robustness and efficiency of text classification.

[0144] In some optional implementations, the low-rank adaptive classification model includes an input layer, a pre-trained super-large language model, a low-rank adaptive network, and multiple classifiers, and the prediction module 304 includes:

[0145] An input submodule, used for preprocessing the training set through the input layer to obtain input text data;

[0146] A semantic feature extraction submodule, used for inputting the input text data into the pre-trained super large language model for semantic feature extraction to obtain a text semantic representation vector;

[0147] A low-rank adaptation submodule, used for performing low-rank decomposition on the input text data through the low-rank adaptation network to obtain a text low-rank adaptation matrix;

[0148] A concatenation submodule, used for concatenating the text semantic representation vector and the text low-rank adaptation matrix to obtain text semantic features;

[0149] The classification submodule is used to input the text semantic features into the multiple classifiers for classification to obtain classification prediction results.

[0150] By training a low-rank adaptive classification model for text classification, we can fully utilize the advantages of the super-large language model in semantic understanding, better capture the information in the text data, and capture the key features of different classification labels through the low-rank adaptive network to achieve the distinction and capture of the correlation between different labels, thereby improving the accuracy and stability of multi-label classification.

[0151] In some optional implementations of this embodiment, the super large language model includes an embedding layer and a multi-level feature extraction network, and the semantic feature extraction submodule is further used to:

[0152] Performing vector conversion on the input text data through the embedding layer to obtain a text embedding vector;

[0153] The text embedding vector is input into the multi-level feature extraction network to perform feature extraction at different scales, so as to obtain a multi-scale fused text semantic representation vector.

[0154] By extracting semantic features through a super-large language model, we can better capture the deep semantic information of the text and identify the classification features in the text data, which in turn helps to effectively capture label relevance and improve the efficiency and accuracy of text classification.

[0155] In some optional implementations of this embodiment, the low-rank adaptive network includes a vector conversion layer and a low-rank adaptive matrix corresponding to each of the classification labels, and the low-rank adaptive submodule is further used for:

[0156] Performing vector conversion on the text in the input text data through the vector conversion layer to obtain a text word vector;

[0157] Input the text word vectors into each of the low-rank adaptation matrices for matrix decomposition to obtain a text feature matrix corresponding to each of the classification labels;

[0158] All the text feature matrices are concatenated to obtain a text low-rank adaptation matrix.

[0159] Low-rank decomposition through a low-rank adaptive network can identify the differences and associations between different classification labels. At the same time, it reduces the number of parameters that need to be adjusted, thereby improving model training efficiency and classification effects.

[0160] In some optional implementations of this embodiment, the classification submodule is further used to:

[0161] Inputting the text semantic features into the multiple classifiers respectively, wherein the multiple classifiers include classifiers corresponding to the respective classification labels;

[0162] Performing classification prediction on the text semantic features through each of the classifiers to obtain a prediction probability of each of the classification labels;

[0163] The classification labels whose prediction probabilities are greater than or equal to a preset confidence threshold are output as classification prediction results.

[0164] By training the classifier, the text multi-label classification task can be completed more quickly and the processing efficiency can be improved. At the same time, the accuracy of text classification can be improved while reducing misjudgments and missed judgments.

[0165] In some optional implementations of this embodiment, the iteration module 305 includes:

[0166] A loss calculation submodule, used to calculate the loss value between the classification prediction result and the classification label according to a preset loss function;

[0167] An adjustment submodule is used to adjust the parameters of the low-rank adaptive network and the classifier based on the loss value by adopting a back-propagation algorithm.

[0168] Through fine-tuning, the model can be better adapted to multi-label classification tasks, reduce training time and computing resources, reduce dependence on large amounts of labeled data, and enhance the model's generalization ability.

[0169] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0170] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0171] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0172] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a text multi-label classification method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0173] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the text multi-label classification method.

[0174] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0175] By preprocessing the original text dataset, the data quality can be improved and the model generalization ability can be enhanced; by obtaining the classification labels of the preprocessed text and marking them, the expressiveness of the classification features can be enhanced, which helps to capture the correlation between different classification labels to achieve accurate multi-label classification; by inputting the training set into the low-rank adaptive classification model and fusing the semantic feature extraction results and the low-rank decomposition results to obtain the classification prediction results, the semantic representation ability of the model can be fully utilized to capture the deep semantic information of the text, and the key information of the text can be identified through low-rank decomposition with fewer parameters, thereby capturing the correlation between different labels, improving the parameter efficiency of the model while improving the accuracy and stability of text multi-label classification.

[0176] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the text multi-label classification method as described above.

[0177] By preprocessing the original text dataset, the data quality can be improved and the model generalization ability can be enhanced; by obtaining the classification labels of the preprocessed text and marking them, the expressiveness of the classification features can be enhanced, which helps to capture the correlation between different classification labels to achieve accurate multi-label classification; by inputting the training set into the low-rank adaptive classification model and fusing the semantic feature extraction results and the low-rank decomposition results to obtain the classification prediction results, the semantic representation ability of the model can be fully utilized to capture the deep semantic information of the text, and the key information of the text can be identified through low-rank decomposition with fewer parameters, thereby capturing the correlation between different labels, improving the parameter efficiency of the model while improving the accuracy and stability of text multi-label classification.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0179] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A text multi-label classification method, characterized in that: The steps include: Acquire an original text data set, and preprocess the original text data set to obtain a preprocessed text data set; Obtaining a classification label of each preprocessed text in the preprocessed text dataset, marking the preprocessed text dataset according to the classification label to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a validation set; Constructing a low-rank adaptive classification model according to the classification labels, and inputting the training set into the low-rank adaptive classification model; Obtaining a semantic feature extraction result and a low-rank decomposition result of the training set through the low-rank adaptive classification model, and fusing the semantic feature extraction result and the low-rank decomposition result to obtain a classification prediction result; Fine-tuning the low-rank adaptive classification model based on the classification prediction result, continuing iterative training until an iteration stop condition is met, and obtaining a fine-tuned classification model; Inputting the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, outputting the fine-tuning classification model as a final text label classification model; The text to be classified is obtained, and the text to be classified is input into the text label classification model for classification to obtain a text classification result.

2. The text multi-label classification method according to claim 1, characterized in that: The step of obtaining the classification label of each preprocessed text in the preprocessed text dataset comprises: The TF-IDF algorithm is used to extract keyword features of each preprocessed text in the preprocessed text dataset; Using a clustering algorithm, classifying the preprocessed text according to the keyword features to obtain different text categories; A classification label for each of the preprocessed texts is generated according to the keyword features of the cluster centers of the text categories.

3. The text multi-label classification method according to claim 1, characterized in that: The low-rank adaptive classification model includes an input layer, a pre-trained super-large language model, a low-rank adaptive network and multiple classifiers. The steps of obtaining the semantic feature extraction result and the low-rank decomposition result of the training set through the low-rank adaptive classification model, and fusing the semantic feature extraction result and the low-rank decomposition result to obtain the classification prediction result include: Preprocessing the training set through the input layer to obtain input text data; Inputting the input text data into the pre-trained super large language model to extract semantic features, and obtaining a text semantic representation vector; Performing low-rank decomposition on the input text data through the low-rank adaptive network to obtain a text low-rank adaptive matrix; Concatenating the text semantic representation vector and the text low-rank adaptation matrix to obtain text semantic features; The text semantic features are respectively input into the multiple classifiers for classification to obtain classification prediction results.

4. The text multi-label classification method according to claim 3 is characterized in that: The super large language model includes an embedding layer and a multi-level feature extraction network, and the step of inputting the input text data into the pre-trained super large language model semantic feature extraction to obtain a text semantic representation vector includes: Performing vector conversion on the input text data through the embedding layer to obtain a text embedding vector; The text embedding vector is input into the multi-level feature extraction network to perform feature extraction at different scales, so as to obtain a multi-scale fused text semantic representation vector.

5. The text multi-label classification method according to claim 3 is characterized in that: The low-rank adaptive network includes a vector conversion layer and a low-rank adaptive matrix corresponding to each of the classification labels; the step of performing low-rank decomposition on the input text data through the low-rank adaptive network to obtain the text low-rank adaptive matrix includes: Performing vector conversion on the text in the input text data through the vector conversion layer to obtain a text word vector; Input the text word vectors into each of the low-rank adaptation matrices for matrix decomposition to obtain a text feature matrix corresponding to each of the classification labels; All the text feature matrices are concatenated to obtain a text low-rank adaptation matrix.

6. The text multi-label classification method according to claim 3, characterized in that: The step of inputting the text semantic features into the multiple classifiers for classification to obtain classification prediction results comprises: Inputting the text semantic features into the multiple classifiers respectively, wherein the multiple classifiers include classifiers corresponding to the respective classification labels; Performing classification prediction on the text semantic features through each of the classifiers to obtain a prediction probability of each of the classification labels; The classification labels whose prediction probabilities are greater than or equal to a preset confidence threshold are output as classification prediction results.

7. The text multi-label classification method according to claim 3 is characterized in that: The step of fine-tuning the low-rank adaptive classification model based on the classification prediction result comprises: Calculate the loss value between the classification prediction result and the classification label according to a preset loss function; Based on the loss value, a back propagation algorithm is used to adjust the parameters of the low-rank adaptive network and the classifier.

8. A text multi-label classification device, characterized in that: include: A preprocessing module, used to obtain an original text data set, preprocess the original text data set, and obtain a preprocessed text data set; A labeling module, used for obtaining a classification label of each preprocessed text in the preprocessed text data set, labeling the preprocessed text data set according to the classification label to obtain a labeled text data set, and randomly dividing the labeled text data set into a training set and a validation set; A construction module, used to construct a low-rank adaptive classification model according to the classification label, and input the training set into the low-rank adaptive classification model; A prediction module, used to obtain a semantic feature extraction result and a low-rank decomposition result of the training set through the low-rank adaptive classification model, and to fuse the semantic feature extraction result and the low-rank decomposition result to obtain a classification prediction result; An iterative module, used for fine-tuning the low-rank adaptive classification model based on the classification prediction result, continuing iterative training until an iteration stop condition is met, and obtaining a fine-tuned classification model; A verification module, used for inputting the verification set into the fine-tuning classification model for verification to obtain a verification result, and when the verification result meets a preset threshold condition, outputting the fine-tuning classification model as a final text label classification model; The classification module is used to obtain the text to be classified, input the text to be classified into the text label classification model for classification, and obtain the text classification result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the text multi-label classification method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the text multi-label classification method according to any one of claims 1 to 7 are implemented.

Images