Text classification method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and finance, and relates to a text classification method, which comprises the following steps: preprocessing an original text data set, labeling classification labels to obtain a labeled text data set, inputting a training set into a large language model for keyword extraction to obtain a candidate keyword set, cleaning the candidate keyword set to obtain a classification keyword set, constructing an instruction template based on the classification keyword set, and inputting the training set and the instruction template into a large language model to obtain a prediction classification result; and based on a prediction classification result and a loss function between the classification labels, performing fine tuning on the large language model to obtain a fine tuning model, evaluating that the fine tuning model is qualified to obtain a text classification model, and inputting the to-be-classified text into the text classification model for classification. The invention further provides a text classification device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the tagged text data set can be stored in a block chain. According to the method, the text classification accuracy and stability can be improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a text classification method, apparatus, computer equipment and storage medium. Background Art

[0002] In the financial field, with the rapid development of big data and artificial intelligence technology, the intelligence level of information processing and decision support systems is increasing. Especially in the context of information explosion, the massive and diversified characteristics of financial data are becoming more and more prominent. How to efficiently and accurately extract valuable information from massive data has become an important challenge in key links such as financial analysis, risk management, and investment decision-making. As the basis for the application of technologies such as information retrieval, natural language processing, and machine learning in the financial field, text classification is of self-evident importance. Through accurate text classification, financial institutions can quickly identify key information such as market trends, customer needs, and news events, providing strong support for business decisions.

[0003] In recent years, the rise of deep learning technology has brought revolutionary breakthroughs in the field of natural language processing. Among them, super-large language models such as the GPT series have achieved remarkable results in multiple tasks such as text generation, dialogue systems, and sentiment analysis with their powerful semantic understanding and generation capabilities. These models are widely used in financial news summaries, automatic generation of market analysis reports, customer service chatbots, etc., greatly improving the intelligence level of financial services and customer satisfaction.

[0004] In the text classification task, the prompt format provides a clear contextual framework for the model through carefully designed example input, thereby guiding the model to generate more accurate and expected classification results. The core of this method is to use the model's understanding of language structure and stimulate the model's potential in specific tasks through the "prompt" effect of examples.

[0005] However, although the prompt format has shown great potential in text classification, its actual application effect is highly dependent on the specific design of the prompt. Different prompt formats may lead to significantly different results in model output, which reflects the sensitivity and importance of prompt format design to model performance. However, the current design of the prompt format mainly relies on manual methods, which is not only time-consuming and labor-intensive, but also easily affected by the designer's subjective judgment and experience level. The process of manually designing the prompt format often lacks unified standards and objective evaluation methods, resulting in differences in the effectiveness and generalization ability of the designed prompt formats, making it difficult to ensure the consistency and accuracy of text classification tasks. Summary of the invention

[0006] The purpose of the embodiments of the present application is to propose a text classification method, apparatus, computer device and storage medium to solve the technical problems in the related art that the design of the text classification prompt format is time-consuming and labor-intensive, and there are differences in effectiveness and generalization ability, resulting in poor consistency and accuracy of text classification.

[0007] In order to solve the above technical problems, the present application embodiment provides a text classification method, which adopts the following technical solution:

[0008] Obtaining an original text dataset, preprocessing and labeling the original text dataset to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a test set;

[0009] Input the training set into a pre-trained large language model, extract keywords according to different classification labels, and obtain a candidate keyword set corresponding to each classification label;

[0010] Performing data cleaning on the candidate keyword set to obtain a final classification keyword set;

[0011] Building an instruction template based on the classification keyword set, inputting the training set and the instruction template into the pre-trained large language model to obtain a predicted classification result;

[0012] Fine-tune the large language model based on the loss function between the predicted classification result and the classification label, and continue iterative optimization until an iteration stop condition is met to obtain a fine-tuned model;

[0013] Evaluating the fine-tuning model according to the test set, and when the evaluation result meets a preset threshold condition, outputting the fine-tuning model as the final text classification model;

[0014] The text to be classified is obtained, and the text to be classified is input into the text classification model to obtain a text classification category.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a text classification device, which adopts the following technical solution:

[0016] The labeling module is used to obtain an original text data set, preprocess and label the original text data set to obtain a labeled text data set, and randomly divide the labeled text data set into a training set and a test set;

[0017] An extraction module is used to input the training set into a pre-trained large language model, extract keywords according to different classification labels, and obtain a candidate keyword set corresponding to each classification label;

[0018] A cleaning module, used for performing data cleaning on the candidate keyword set to obtain a final classification keyword set;

[0019] A prediction module, used for constructing an instruction template based on the classification keyword set, inputting the training set and the instruction template into the pre-trained large language model, and obtaining a prediction classification result;

[0020] An iteration module, configured to fine-tune the large language model based on a loss function between the predicted classification result and the classification label, and continue iterative optimization until an iteration stop condition is met to obtain a fine-tuned model;

[0021] An evaluation module, used for evaluating the fine-tuning model according to the test set, and outputting the fine-tuning model as a final text classification model when the evaluation result meets a preset threshold condition;

[0022] The classification module is used to obtain the text to be classified, input the text to be classified into the text classification model, and obtain the text classification category.

[0023] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0024] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned text classification method when executing the computer-readable instructions.

[0025] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0026] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the text classification method described above are implemented.

[0027] Compared with the prior art, this application has the following beneficial effects:

[0028] The present application provides a text classification method, which preprocesses and labels a raw text data set to obtain a labeled text data set, and randomly divides the labeled text data set into a training set and a test set, inputs the training set into a pre-trained large language model, extracts keywords according to different classification labels, obtains a candidate keyword set corresponding to each classification label, performs data cleaning on the candidate keyword set, obtains a final classification keyword set, constructs an instruction template based on the classification keyword set, inputs the training set and the instruction template into the pre-trained large language model, obtains a predicted classification result; fine-tunes the large language model based on a loss function between the predicted classification result and the classification label, and continues to iterate and optimize until Until the iteration stopping condition is met, a fine-tuning model is obtained, and the fine-tuning model is evaluated according to the test set. When the evaluation result meets the preset threshold condition, the fine-tuning model is output as the final text classification model, and the text to be classified is input into the text classification model to obtain the text classification category; keyword extraction of text data through a pre-trained large language model can improve the efficiency and accuracy of key extraction, and model training through the pre-trained large language model combined with the extracted keywords and training set can improve the model's ability to understand text data, better capture the classification information in text data, enhance the generalization ability of the model, improve the accuracy and stability of text classification, and be more adaptable to complex and diverse text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0031] Figure 2 is a flowchart of an embodiment of a text classification method according to the present application;

[0032] Figure 3 is a structural schematic diagram of an embodiment of a text classification device according to the present application;

[0033] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0037] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0038] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0039] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0040] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0041] It should be noted that the text classification method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the text classification device is generally set in the server / terminal device.

[0042] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0043] Continue to refer Figure 2 , shows a flowchart of an embodiment of a text classification method according to the present application, comprising the following steps:

[0044] Step S201, obtaining an original text dataset, preprocessing and labeling the original text dataset with classification labels to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a test set.

[0045] According to the text classification scenario, the original text dataset corresponding to the business scenario is obtained. The original text dataset contains a large amount of original text data, which can be unlabeled text data, partially labeled text data, or text data containing noise. The source of the original text dataset includes but is not limited to financial or trading platforms, public datasets dedicated to the financial field, business databases, etc.

[0046] In this embodiment, the text classification method is executed on the electronic device (eg Figure 1The server / terminal device shown in the figure can receive the acquired original text data set through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0047] The original text dataset is preprocessed and labeled with classification labels to obtain a labeled text dataset marked with classification labels. The data in the labeled text dataset exists in the form of [label, text].

[0048] In some optional implementations of this embodiment, the steps of preprocessing the original text dataset and labeling the classification labels to obtain the labeled text dataset include:

[0049] Performing a preprocessing operation on the original text data in the original text data set to obtain a preprocessed text data set;

[0050] The TF-IDF algorithm is used to extract keyword features from the preprocessed text dataset;

[0051] Through the clustering algorithm, the preprocessed text data set will be classified according to the keyword features to obtain multiple text classification categories;

[0052] Generate classification labels based on the keyword features of the cluster center of the text classification category;

[0053] The text data in each text classification category is associated and annotated with the corresponding classification label to obtain an annotated text dataset containing text data and classification labels.

[0054] In this embodiment, the preprocessing operation includes denoising, word segmentation, etc., and denoising includes removing irrelevant characters and special symbols, removing stop words, spelling checking, grammar correction, removing duplicate content, etc. Specifically, the original text data in the original text data set is denoised to obtain denoised text data; the denoised text data is segmented by word segmentation technology to obtain a preprocessed text data set containing word segmentation units.

[0055] The TF-IDF algorithm is used to calculate the importance weight of each word segmentation unit in the preprocessed text dataset, that is, the ratio of the frequency of occurrence of the word segmentation unit in the text in which it is located to the frequency of occurrence in all texts; the word segmentation units whose importance weight is greater than or equal to the preset weight threshold are screened out as the keyword features of the corresponding text.

[0056] In this embodiment, a K-means clustering algorithm is used to classify text data in a preprocessed text data set. Specifically, K initial cluster centers are determined according to keyword features through the K-means clustering algorithm, where K is a positive integer; all keyword features are traversed, the distance between the keyword features and each initial cluster center is calculated, and the group corresponding to the keyword features is determined; the cluster center of each group is updated to obtain the actual cluster center of each group; the distance between the keyword features and each actual cluster center is repeatedly calculated, and the cluster center of each group is continuously updated until the group no longer changes, thereby obtaining different keyword categories; the similarity scores between all keyword features of each text data and each keyword category are calculated, and the keyword category with the highest similarity score is used as the text classification category of the text data.

[0057] The classification labels corresponding to the text classification categories are automatically generated according to the keyword features corresponding to the cluster center of each text classification category. Professional text annotation tools such as LabelImg, Labelme, etc. are used to annotate the text data with classification labels to obtain an annotated text dataset containing text data and classification labels.

[0058] By preprocessing and labeling the original text dataset, the data quality can be improved and the expressiveness of features can be enhanced, so as to improve the generalization ability and training speed of the model, thereby improving the accuracy, robustness and efficiency of text classification.

[0059] In this embodiment, the annotated text data set is randomly divided into a training set and a test set according to a preset ratio. The preset ratio can be set according to actual needs. For example, the preset ratio is 8:2, that is, the ratio of the training set to the test set is 8:2.

[0060] It should be emphasized that in order to further ensure the privacy and security of the annotated text dataset, the above-mentioned annotated text dataset can also be stored in a node of a blockchain.

[0061] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0062] Step S202: input the training set into the pre-trained large language model, extract keywords according to different classification labels, and obtain a set of candidate keywords corresponding to each classification label.

[0063] In this embodiment, a pre-trained large language model is used to extract keywords of each text classification category according to the classification label. The large language model is a super large language model. Specifically, a pre-trained large language model is obtained as a basic model; for the input training set, the classification label types contained in the training set are determined to determine the number of labels required to generate keywords; each classification label in the training set is traversed, and according to the current classification label, a batch of candidate keywords, i.e., a candidate keyword set, is generated using the pre-trained large language model.

[0064] In some optional implementations, the pre-trained large language model includes an embedding layer, an encoding network, a decoding network, and an extraction output layer connected in sequence, and the step of inputting the training set into the pre-trained large language model, extracting keywords according to different classification labels, and obtaining a candidate keyword set corresponding to each classification label includes:

[0065] The training set is input into the pre-trained large language model, and the text data in the category data set is converted into vectors through the embedding layer to obtain the embedded vector;

[0066] The feature extraction of the embedding vector is performed through the attention mechanism of the encoding network to obtain the text classification feature vector;

[0067] The text classification feature vector is input into the decoding network, and autoregression is performed according to the classification label to obtain the predicted keyword sequence for each classification label;

[0068] The predicted keyword sequence is input into the extraction output layer for sequence decoding to obtain a set of candidate keywords.

[0069] The business sample data is preprocessed through the embedding layer, including word segmentation, part-of-speech tagging, etc., and then the preprocessed text data is converted into an embedding vector that the model can understand. Vector conversion includes position embedding, word embedding, and character embedding of text data. Position embedding provides information about the position of words in the sentence, which helps to understand the grammatical structure and contextual relationship of the sentence; word embedding captures the semantic information of the vocabulary, allowing the model to understand the meaning of the vocabulary in the context; character embedding captures character-level information, which helps the model process unknown vocabulary or spelling variants. By incorporating character-level information into the model, the model's ability to process complex data is improved. The embedding vector is the concatenation of the position embedding vector, word embedding vector, and character embedding vector.

[0070] In this embodiment, the encoding network extracts the semantic and contextual features of the text embedding vector through a multi-head attention mechanism to obtain a text classification feature vector. In some optional implementations, the encoding network uses multiple stacked connected encoders, and encodes the embedding vector in turn through multiple layers of encoders to extract the semantic and contextual features of the text data. Among them, the input of the latter encoder is the output and embedding vector of the previous encoder. After the processing of the encoder, a deep semantic feature representation of the text data can be obtained, and the output of the last layer of encoder is usually used as the text classification feature vector. In a specific example, each encoder includes a multi-head self-attention layer and a feedforward network layer, and the embedding vector is input into the multi-head self-attention layer for weight calculation to obtain an attention fusion feature, wherein different attention heads in the multi-head self-attention layer process text data of different scales, such as character scale, phrase scale, sentence scale, etc.; the attention fusion feature is feature optimized by the feedforward network layer to obtain a text classification feature vector.

[0071] Next, the text classification feature vector is autoregressively generated through the decoding network to obtain a predicted keyword sequence. In some optional implementations, the decoding network includes a position embedding layer and multiple connected decoders, and each step of keyword generation in the decoding network predicts the next keyword based on the previously generated keyword and the output of the encoder. Among them, each decoder includes a masked multi-head self-attention layer, an encoder-decoder attention layer, and a feedforward network layer. The masked multi-head self-attention layer performs a multi-head attention mechanism calculation on the input text classification feature vector to obtain a masked attention feature; the encoder-decoder attention layer performs a cross-attention mechanism calculation on the masked attention feature to obtain a global attention feature; the global attention feature is input into the feedforward network layer for keyword generation to obtain a predicted keyword sequence.

[0072] The predicted keyword sequence is input into the extraction output layer for sequence decoding, and sequence decoding technology such as BeamSearch (beam search algorithm) is used to control the quantity and quality of generated keywords. For example, if the Beam Size is set to 10, 10 candidate keywords are generated for each classification label, and the candidate keyword set corresponding to each classification label is obtained.

[0073] Keyword extraction through a pre-trained large language model can provide efficient and accurate keyword recognition capabilities, generate keywords that are more in line with each classification label, and improve the efficiency and accuracy of keyword extraction.

[0074] Step S203: perform data cleaning on the candidate keyword set to obtain a final classification keyword set.

[0075] Since the output results of the large language model are relatively messy, it is necessary to clean the output candidate keyword set to obtain the final keywords.

[0076] In this embodiment, the candidate keyword set generated by each classification tag is deduplicated and screened to obtain the final classification keyword set. Specifically, a string matching algorithm is used to determine whether there are repeated keywords in the candidate keyword set. If there are duplicates, deduplication is performed to obtain a deduplicated keyword set; the semantic similarity between each candidate keyword in the deduplicated keyword set and the corresponding classification tag is calculated to obtain a similarity score; according to a preset relevance threshold, candidate keywords with similarity scores greater than or equal to the relevance threshold are screened to form the final classification keyword set.

[0077] For example, the candidate keywords in the candidate keyword set are converted into a string form, and the KMP string matching algorithm is used to traverse all the candidate keywords to determine whether the string of the current candidate keyword is repeated in the subsequent keyword string; if the current keyword string is repeated, the keyword string is marked as a repeated keyword, and its corresponding candidate keyword is removed from the candidate keyword set. After the traversal is completed, a deduplicated keyword set is obtained; the cosine similarity algorithm is used to calculate the semantic similarity between each candidate keyword in the deduplicated keyword set and the corresponding classification label, and the similarity score of all the candidate keywords is obtained; the similarity score is sorted from high to low, and the candidate keywords with a similarity score greater than or equal to the relevance threshold are screened out to form a final classification keyword set.

[0078] In some optional implementations, the filtered classification keyword set is associated with the corresponding classification label and stored in a keyword mapping table.

[0079] By performing data cleaning on the candidate keyword set and removing irrelevant keywords, we can ensure the matching and coverage of keywords and classification labels and improve the overall classification effect.

[0080] Step S204, constructing an instruction template based on the classification keyword set, inputting the training set and the instruction template into a pre-trained large language model to obtain a predicted classification result.

[0081] In this embodiment, a command template based on classification clues, namely a prompt template, is constructed with keywords. The prompt template is a prompt suitable for a specific text classification task. These prompts will guide the model to map the input text to the correct classification category, thereby achieving more accurate text classification.

[0082] The instruction template is constructed based on the classification keyword set. The instruction template contains elements such as input text, classification clues and label placeholders. Among them, the input text is the input text data, the classification clues are the classification keywords, and the label placeholders are the classification labels. The corresponding information is mapped to the corresponding position of the instruction template, that is, the filled instruction template is obtained. The pre-trained large language model is used to perform text classification on the filled instruction template. In the classification process, by integrating information such as classification clues, the model is guided to perform text classification and optimization in a specific direction.

[0083] In some optional implementations, the pre-trained large language model further includes a classification output layer connected to the encoding network, and the training set and the instruction template are input into the pre-trained large language model to obtain a predicted classification result, including:

[0084] Map the text data of the training set to the corresponding position of the instruction template to generate classification prompt text;

[0085] The classification prompt text is converted into a vector through the embedding layer to obtain the prompt text vector;

[0086] The feature extraction of the prompt text vector is performed through the attention mechanism of the encoding network to obtain the semantic features of the prompt text;

[0087] The semantic features of the prompt text are input into the classification output layer for classification prediction to obtain the predicted classification results.

[0088] In this embodiment, the pre-trained large language model is used to extract keywords from the input text and classify the input text according to the extracted keywords. Depending on the task, the pre-trained large language model is divided into two branches, one is a keyword extraction branch composed of a decoding network and an extraction output layer, and the other is a text classification branch formed by a classification output layer. The two branches are respectively connected to the encoding network.

[0089] For the input training set, the text data of the training set is tokenized and mapped, and the input text information is filled in the corresponding position of the instruction template to generate the classification prompt text. The classification prompt text is converted into a vector through the embedding layer to obtain the prompt text vector; the prompt text vector is weighted by the multi-head attention mechanism of the encoding network to capture the context information and obtain the semantic features of the prompt text; the probability distribution of each classification label is predicted through the classification output layer, and the classification label with the highest probability value is selected as the final predicted classification result based on the probability distribution. For example, the classification output layer uses a fully connected layer, the semantic features of the prompt text are input into the fully connected layer, and the semantic features of the prompt text are mapped to the predefined classification label through the Softmax activation function to obtain the classification probability distribution of each classification label, and the predicted classification result is obtained according to the classification probability distribution.

[0090] By fusing input text and keywords with a pre-trained large language model for text classification, we can better capture the classification information in text data and improve the efficiency and accuracy of text classification.

[0091] Step S205, fine-tuning the large language model based on the loss function between the predicted classification result and the classification label, and continuing the iterative optimization until the iteration stop condition is met to obtain a fine-tuned model.

[0092] Specifically, the loss value between the predicted classification result and the classification label is calculated according to the preset cross entropy loss function; through the back propagation algorithm, the gradient of the loss value to the model parameters is returned to update the network parameters of the extraction output layer and the classification output layer.

[0093] In this embodiment, the preset loss function can select the cross entropy loss function, which can measure the difference between the model prediction result and the true label. The loss value between the predicted classification result and the classification label is calculated, and the gradient of the loss value to the model parameter is returned through the back propagation algorithm to adjust the network parameters of the extraction output layer and the classification output layer to realize the model fine-tuning process. Use the fine-tuned model to re-extract keywords, and repeat steps S203 to S205 until the iteration stop condition is met to obtain a converged fine-tuning model. Among them, the iteration stop condition is to reach the preset maximum number of iterations, or the current loss value no longer decreases compared to the loss value of the previous round.

[0094] By continuously re-extracting keywords and constructing instruction templates during the fine-tuning process, the instruction templates that guide the model for text classification are continuously optimized. While reducing the demand for labeled data, its generalization ability and robustness are continuously improved, enabling the model to better complete text classification tasks, significantly improving the performance and efficiency of text classification tasks, and better adapting to actual application scenarios.

[0095] Step S206, evaluating the fine-tuning model according to the test set, and when the evaluation result meets the preset threshold condition, outputting the fine-tuning model as the final text classification model.

[0096] In the model evaluation phase, the test set is input into the fine-tuning model to obtain the test classification results of the text classification of the test set by the model; according to the preset evaluation indicators, the evaluation results between the test classification results and the corresponding true classification labels are calculated. When the evaluation results are greater than or equal to the preset evaluation threshold, that is, the evaluation results meet the preset threshold conditions, indicating that the model meets expectations, the current fine-tuning model is output as the final text classification model. Among them, the preset evaluation indicators include accuracy, precision, recall rate, F1 value, etc. The actual performance of the model can be comprehensively judged by the evaluation indicators, which can improve the accuracy of the model's text classification and model performance.

[0097] In some optional implementations, after the step of evaluating the fine-tuning model according to the test set, the method further includes:

[0098] When the evaluation result does not meet the preset threshold condition, filter out the sample data with incorrect classification in the test set;

[0099] Optimize and adjust the fine-tuning model according to the sample data to obtain an optimized model;

[0100] Use the optimized model to reclassify the sample data and evaluate the classification effect.

[0101] When the evaluation result is less than the preset evaluation threshold, it means that the evaluation result does not meet the preset threshold condition. Then, the sample data with incorrect classification in the test set is screened out according to the evaluation result; the screened out erroneous samples are analyzed to determine the causes of the classification errors and obtain the analysis results; based on the analysis results, the fine-tuning model is further optimized and adjusted; the optimized model is used to re-classify the text of the sample data with incorrect classification, and the optimized classification effect is evaluated.

[0102] Specifically, for sample data that is misclassified, its corresponding classification keyword set is re-acquired, and the prompt template is updated according to the classification keyword set; the updated prompt template and sample data are input into the fine-tuning model; the updated prompt template is used to perform further fine-tuning on the fine-tuning model; the classification accuracy of the model after the further fine-tuning is obtained; and it is determined whether the classification accuracy reaches a preset accuracy threshold. If so, the model after the further fine-tuning is output as the final text classification model. If not, the step returns to step S202 to re-extract keywords and continue to obtain the classification keyword set of the sample data that is misclassified.

[0103] By continuing to optimize and adjust the models whose evaluation results do not meet the preset threshold conditions, the robustness and practicality of the model can be ensured, thereby improving the accuracy of text classification.

[0104] Step S207, obtaining the text to be classified, inputting the text to be classified into a text classification model, and obtaining a text classification category.

[0105] The trained text classification model is used to perform keyword extraction and text classification on the text to be classified to obtain the text classification category.

[0106] In this embodiment, the text classification model is adjusted and optimized regularly to obtain a text classification model with stronger generalization ability and robustness to adapt to complex and changing financial business scenarios.

[0107] This application uses a pre-trained large language model to extract keywords from text data, which can improve the efficiency and accuracy of key extraction. The pre-trained large language model is combined with the extracted keywords and training sets to perform model training, which can improve the model's ability to understand text data, better capture information in text data, enhance the model's generalization ability, improve the accuracy and stability of text classification, and be more adaptable to complex and diverse text data.

[0108] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0109] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0111] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0112] Further references Figure 3 , as a response to the above Figure 2The present application provides an embodiment of a text classification device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] like Figure 3 As shown, the text classification device 300 described in this embodiment includes: a labeling module 301, an extraction module 302, a cleaning module 303, a prediction module 304, an iteration module 305, an evaluation module 306 and a classification module 307. Among them:

[0114] The labeling module 301 is used to obtain an original text data set, preprocess and label the original text data set to obtain a labeled text data set, and randomly divide the labeled text data set into a training set and a test set;

[0115] The extraction module 302 is used to input the training set into a pre-trained large language model, extract keywords according to different classification labels, and obtain a candidate keyword set corresponding to each classification label;

[0116] The cleaning module 303 is used to perform data cleaning on the candidate keyword set to obtain a final classification keyword set;

[0117] The prediction module 304 is used to construct an instruction template based on the classification keyword set, input the training set and the instruction template into the pre-trained large language model, and obtain a prediction classification result;

[0118] The iteration module 305 is used to fine-tune the large language model based on the loss function between the predicted classification result and the classification label, and continue iterative optimization until an iteration stop condition is met to obtain a fine-tuned model;

[0119] The evaluation module 306 is used to evaluate the fine-tuning model according to the test set, and when the evaluation result meets the preset threshold condition, output the fine-tuning model as the final text classification model;

[0120] The classification module 307 is used to obtain the text to be classified, input the text to be classified into the text classification model, and obtain the text classification category.

[0121] It should be emphasized that in order to further ensure the privacy and security of the annotated text dataset, the above-mentioned annotated text dataset can also be stored in a node of a blockchain.

[0122] Based on the above-mentioned text classification device 300, keyword extraction of text data is performed through a pre-trained large language model, which can improve the efficiency and accuracy of key extraction, and model training is performed through the pre-trained large language model combined with the extracted keywords and training sets, which can improve the model's ability to understand text data, better capture classification information in text data, enhance the generalization ability of the model, improve the accuracy and stability of text classification, and be more adaptable to complex and diverse text data.

[0123] In some optional implementations, the marking module 301 is further used to:

[0124] Performing a preprocessing operation on the original text data in the original text data set to obtain a preprocessed text data set;

[0125] Using TF-IDF algorithm to extract keyword features of the preprocessed text data set;

[0126] By using a clustering algorithm, the preprocessed text data set is classified according to the keyword features to obtain a plurality of text classification categories;

[0127] Generating a classification label according to the keyword features of the cluster center of the text classification category;

[0128] The text data in each of the text classification categories is associated and annotated with the corresponding classification label to obtain an annotated text data set including the text data and the classification label.

[0129] By preprocessing and labeling the original text dataset, the data quality can be improved and the expressiveness of features can be enhanced, so as to improve the generalization ability and training speed of the model, thereby improving the accuracy, robustness and efficiency of text classification.

[0130] In some optional implementations, the pre-trained large language model includes an embedding layer, an encoding network, a decoding network, and an extraction output layer connected in sequence, and the extraction module 302 includes:

[0131] An embedding submodule, used for inputting the training set into a pre-trained large language model, performing vector conversion on the text data in the training set through the embedding layer, and obtaining an embedding vector;

[0132] An attention submodule, configured to extract features from the embedding vector through the attention mechanism of the encoding network to obtain a text classification feature vector;

[0133] A decoding submodule, used for inputting the text classification feature vector into the decoding network, performing autoregressive generation according to the classification label, and obtaining a predicted keyword sequence for each classification label;

[0134] The output submodule is used to input the predicted keyword sequence into the extraction output layer for sequence decoding to obtain a candidate keyword set.

[0135] Keyword extraction through a pre-trained large language model can provide efficient and accurate keyword recognition capabilities, generate keywords that are more in line with each classification label, and improve the efficiency and accuracy of keyword extraction.

[0136] In some optional implementations of this embodiment, the cleaning module 303 is further used to:

[0137] A string matching algorithm is used to determine whether there are repeated keywords in the candidate keyword set, and if there are repeated keywords, duplicate removal is performed to obtain a duplicate-free keyword set;

[0138] Calculating the semantic similarity between each candidate keyword in the deduplicated keyword set and the corresponding classification label to obtain a similarity score;

[0139] According to a preset relevance threshold, candidate keywords with similarity scores greater than or equal to the relevance threshold are screened out to form a final classification keyword set.

[0140] By performing data cleaning on the candidate keyword set and removing irrelevant keywords, we can ensure the matching and coverage of keywords and classification labels and improve the overall classification effect.

[0141] In some optional implementations, the pre-trained large language model further includes a classification output layer connected to the encoding network, and the prediction module 304 is further used to:

[0142] Mapping the text data of the training set to the corresponding position of the instruction template to generate classification prompt text;

[0143] The classification prompt text is vectorized by the embedding layer to obtain a prompt text vector;

[0144] Extracting features of the prompt text vector through the attention mechanism of the encoding network to obtain semantic features of the prompt text;

[0145] The semantic features of the prompt text are input into the classification output layer for classification prediction to obtain a predicted classification result.

[0146] By fusing input text and keywords with a pre-trained large language model for text classification, we can better capture the classification information in text data and improve the efficiency and accuracy of text classification.

[0147] In this embodiment, the iteration module 305 is further used for:

[0148] Calculating the loss value between the predicted classification result and the classification label according to a preset cross entropy loss function;

[0149] Through the back-propagation algorithm, the gradient of the loss value to the model parameters is returned to update the network parameters of the extraction output layer and the classification output layer.

[0150] By continuously re-extracting keywords and constructing instruction templates during the fine-tuning process, the instruction templates that guide the model for text classification are continuously optimized. While reducing the demand for labeled data, its generalization ability and robustness are continuously improved, enabling the model to better complete text classification tasks, significantly improving the performance and efficiency of text classification tasks.

[0151] In some optional implementations of this embodiment, the text classification device 300 further includes an optimization module, which is used to:

[0152] When the evaluation result does not meet the preset threshold condition, filter out the sample data that is misclassified in the test set;

[0153] Optimizing and adjusting the fine-tuning model according to the sample data to obtain an optimized model;

[0154] The optimized model is used to reclassify the sample data and evaluate the classification effect.

[0155] By continuing to optimize and adjust the models whose evaluation results do not meet the preset threshold conditions, the robustness and practicality of the model can be ensured, thereby improving the accuracy of text classification.

[0156] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0157] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0158] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0159] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the text classification method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0160] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the text classification method.

[0161] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0162] Extracting keywords from text data through a pre-trained large language model can improve the efficiency and accuracy of key extraction. Training the model by combining the extracted keywords with the training set using a pre-trained large language model can improve the model's ability to understand text data, better capture classification information in text data, enhance the model's generalization ability, improve the accuracy and stability of text classification, and be more adaptable to complex and diverse text data.

[0163] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the text classification method as described above.

[0164] Extracting keywords from text data through a pre-trained large language model can improve the efficiency and accuracy of key extraction. Training the model by combining the extracted keywords with the training set using a pre-trained large language model can improve the model's ability to understand text data, better capture classification information in text data, enhance the model's generalization ability, improve the accuracy and stability of text classification, and be more adaptable to complex and diverse text data.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0166] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A text classification method, characterized in that: The steps include: Obtaining an original text dataset, preprocessing and labeling the original text dataset to obtain a labeled text dataset, and randomly dividing the labeled text dataset into a training set and a test set; Input the training set into a pre-trained large language model, extract keywords according to different classification labels, and obtain a candidate keyword set corresponding to each classification label; Performing data cleaning on the candidate keyword set to obtain a final classification keyword set; Building an instruction template based on the classification keyword set, inputting the training set and the instruction template into the pre-trained large language model to obtain a predicted classification result; Fine-tune the large language model based on the loss function between the predicted classification result and the classification label, and continue iterative optimization until an iteration stop condition is met to obtain a fine-tuned model; Evaluating the fine-tuning model according to the test set, and when the evaluation result meets a preset threshold condition, outputting the fine-tuning model as the final text classification model; The text to be classified is obtained, and the text to be classified is input into the text classification model to obtain a text classification category.

2. The text classification method according to claim 1, characterized in that: The step of preprocessing the original text data set and labeling the classification labels to obtain the labeled text data set includes: Performing a preprocessing operation on the original text data in the original text data set to obtain a preprocessed text data set; Using TF-IDF algorithm to extract keyword features of the preprocessed text data set; By using a clustering algorithm, the preprocessed text data set is classified according to the keyword features to obtain a plurality of text classification categories; Generating a classification label according to the keyword features of the cluster center of the text classification category; The text data in each of the text classification categories is associated and annotated with the corresponding classification label to obtain an annotated text data set including the text data and the classification label.

3. The text classification method according to claim 1, characterized in that: The pre-trained large language model includes an embedding layer, an encoding network, a decoding network and an extraction output layer connected in sequence. The step of inputting the training set into the pre-trained large language model, extracting keywords according to different classification labels, and obtaining a candidate keyword set corresponding to each classification label includes: Inputting the training set into a pre-trained large language model, performing vector conversion on the text data in the training set through the embedding layer to obtain an embedding vector; Extracting features from the embedding vector through the attention mechanism of the encoding network to obtain a text classification feature vector; Inputting the text classification feature vector into the decoding network, performing autoregressive generation according to the classification label, and obtaining a predicted keyword sequence for each classification label; The predicted keyword sequence is input into the extraction output layer for sequence decoding to obtain a candidate keyword set.

4. The text classification method according to claim 1, characterized in that: The step of performing data cleaning on the candidate keyword set to obtain a final classification keyword set includes: A string matching algorithm is used to determine whether there are repeated keywords in the candidate keyword set, and if there are repeated keywords, duplicate removal is performed to obtain a duplicate-free keyword set; Calculating the semantic similarity between each candidate keyword in the deduplicated keyword set and the corresponding classification label to obtain a similarity score; According to a preset relevance threshold, candidate keywords with similarity scores greater than or equal to the relevance threshold are screened out to form a final classification keyword set.

5. The text classification method according to claim 3, characterized in that: The pre-trained large language model further includes a classification output layer connected to the encoding network, and the step of inputting the training set and the instruction template into the pre-trained large language model to obtain a predicted classification result includes: Mapping the text data of the training set to the corresponding position of the instruction template to generate classification prompt text; The classification prompt text is vectorized by the embedding layer to obtain a prompt text vector; Extracting features of the prompt text vector through the attention mechanism of the encoding network to obtain semantic features of the prompt text; The semantic features of the prompt text are input into the classification output layer for classification prediction to obtain a predicted classification result.

6. The text classification method according to claim 5, characterized in that: The step of fine-tuning the large language model based on the loss function between the predicted classification result and the classification label comprises: Calculating the loss value between the predicted classification result and the classification label according to a preset cross entropy loss function; Through the back-propagation algorithm, the gradient of the loss value to the model parameters is returned to update the network parameters of the extraction output layer and the classification output layer.

7. The text classification method according to claim 1, characterized in that: After the step of evaluating the fine-tuning model according to the test set, the method further includes: When the evaluation result does not meet the preset threshold condition, filter out the sample data that is misclassified in the test set; Optimizing and adjusting the fine-tuning model according to the sample data to obtain an optimized model; The optimized model is used to reclassify the sample data and evaluate the classification effect.

8. A text classification device, characterized in that: include: The labeling module is used to obtain an original text data set, preprocess and label the original text data set to obtain a labeled text data set, and randomly divide the labeled text data set into a training set and a test set; An extraction module is used to input the training set into a pre-trained large language model, extract keywords according to different classification labels, and obtain a candidate keyword set corresponding to each classification label; A cleaning module, used for performing data cleaning on the candidate keyword set to obtain a final classification keyword set; A prediction module, used for constructing an instruction template based on the classification keyword set, inputting the training set and the instruction template into the pre-trained large language model, and obtaining a prediction classification result; An iteration module, configured to fine-tune the large language model based on a loss function between the predicted classification result and the classification label, and continue iterative optimization until an iteration stop condition is met to obtain a fine-tuned model; An evaluation module, used for evaluating the fine-tuning model according to the test set, and outputting the fine-tuning model as a final text classification model when the evaluation result meets a preset threshold condition; The classification module is used to obtain the text to be classified, input the text to be classified into the text classification model, and obtain the text classification category.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the text classification method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the text classification method according to any one of claims 1 to 7 are implemented.

Images