Domain knowledge fusion data enhancement method for text classification

Abstract

The invention provides a domain knowledge fusion data enhancement method for text classification, and relates to the technical field of data enhancement. After a target domain literature library is connected, a basic corpus is extracted and constructed from the target domain literature library according to a text classification task, and a data enhancement prompt word is defined; the method comprises the steps of inputting cue words and a basic corpus into a data enhancement large language model for data enhancement to generate an enhanced corpus, constructing an instruction data set, performing fusion feedback learning on the data enhancement large language model by adopting a low-rank self-adaptive technology, obtaining an optimized enhanced corpus according to an optimized model, and then obtaining a classification result. The problem that the generated sample is insufficient in semantic depth and field adaptability due to the fact that domain knowledge cannot be effectively combined for deep text data enhancement is solved, the text sample with high quality and high field adaptability is effectively generated, the accuracy and generalization ability of a text classification model are improved, and the text classification efficiency is improved. And the availability of the generated data is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data enhancement, and in particular to a domain knowledge fusion data enhancement method for text classification. Background Art

[0002] Data augmentation is a common technique in machine learning and deep learning to improve the generalization ability of models, especially under conditions of data scarcity. It increases the diversity and size of the training set by modifying existing data samples or generating new data samples. In the field of computer vision, data augmentation includes operations such as rotation, flipping, and scaling; in the field of natural language processing (NLP), it includes techniques such as synonym replacement, back translation, and sentence rearrangement to generate new text that is grammatically correct and semantically consistent.

[0003] Although data augmentation has been widely used to improve model performance, existing text data augmentation techniques mainly focus on simple text operations, such as synonym replacement or rule-based text transformation, which often fail to fully consider the language characteristics and deep semantic structure of specific fields. In addition, these methods have limitations in the diversity and quality of generated text samples, which may result in the enhanced data being unable to effectively cover the complexity of the field, thereby limiting the effectiveness and robustness of the model in practical applications.

[0004] In summary, the existing technology cannot effectively combine domain knowledge for deep text data enhancement, and the data richness and coverage are poor, resulting in the generated samples being insufficient in semantic depth and domain adaptability, and the quality and availability of the generated data are limited. Summary of the invention

[0005] The present application provides a domain knowledge fusion data enhancement method for text classification, which is used to solve the technical problems that the existing technology cannot effectively combine domain knowledge for deep text data enhancement, the data richness and coverage are poor, resulting in the generated samples being insufficient in semantic depth and domain adaptability, and the generated data quality and availability are limited.

[0006] The present application provides a method for domain knowledge fusion data enhancement for text classification, the method comprising:

[0007] The target domain document library is connected, and text is extracted from the target domain document library according to the text classification task to construct a basic corpus; data enhancement prompt words are defined according to the target text classification task, and the data enhancement prompt words include task description, seed sentences and generation requirements; the data enhancement prompt words and the basic corpus are input into the data enhancement large language model for data enhancement to generate an enhanced corpus; an instruction data set is constructed, and based on the instruction data set, low-rank adaptive technology is used to perform fusion feedback learning on the data enhancement large language model to obtain an optimized data enhancement large language model; based on the optimized data enhancement large language model, an optimized enhanced corpus is obtained, and the classification result of the text classification task is obtained based on the optimized enhanced corpus.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] The domain knowledge fusion data enhancement method for text classification provided by the present application is to connect a target domain document library, extract from the target domain document library according to a text classification task, and construct a basic corpus; define data enhancement prompt words according to the target text classification task, and the data enhancement prompt words include a task description, a seed sentence, and a generation requirement; input the data enhancement prompt words and the basic corpus into a data enhancement large language model for data enhancement to generate an enhanced corpus; construct an instruction data set, and use a low-rank adaptive technology based on the instruction data set to perform fusion feedback learning on the data enhancement large language model to obtain an optimized data enhancement large language model; obtain an optimized enhanced corpus according to the optimized data enhancement large language model, and obtain the classification result of the text classification task according to the optimized enhanced corpus, which solves the technical problems that the prior art cannot effectively combine domain knowledge for deep text data enhancement, the richness and coverage of data are poor, resulting in insufficient semantic depth and domain adaptability of the generated samples, and the quality and availability of the generated data are limited, and achieves the technical effect of effectively generating high-quality and domain-adaptable text samples, improving the accuracy and generalization ability of the text classification model, and enhancing the availability of the generated data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a domain knowledge fusion data enhancement method for text classification is provided for this application.

[0011] Figure 2 A flowchart of constructing an instruction dataset in a domain knowledge fusion data enhancement method for text classification is provided for this application. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0013] In order to make the application scheme more comprehensive, the implementation of data enhancement and its application scenarios in text classification tasks are explained below:

[0014] At present, data enhancement in natural language processing mainly uses linguistic knowledge and statistical methods to expand and enrich training data. The main process is as follows: First, the original text data is linguistically analyzed and understood. Then, based on predefined language rules and corpus statistics, the parts of the text that can be replaced, reorganized or converted are identified. Next, specific enhancement operations are performed, usually including methods such as synonym replacement, back translation and sentence reorganization. Among them, in the synonym replacement process, according to the context and semantics of the words, suitable candidate words are selected from the synonym library for replacement to ensure that the replaced sentences retain the original meaning. In the back translation process, the original text is first translated into another language, and then translated back to the original language, thereby generating sentences that express the same meaning but with different wording. The sentence reorganization method mainly creates new expressions by adjusting the sentence structure or word order while maintaining the original semantics.

[0015] As a key technology in text classification tasks, data augmentation plays an important role in improving model performance and solving data-related problems. As an effective means of expanding training data, it can significantly improve the learning process of the model. Specifically, in existing text classification tasks, the performance of the model is usually directly related to the amount of available training data. By applying enhancement methods such as synonym replacement, back translation or sentence reorganization, the model can generate a large number of new training samples based on limited original data. These newly generated samples provide diversified expressions while maintaining the original semantics, thereby expanding the learning scope of the model. Secondly, data augmentation can improve the generalization ability of the model and help improve the accuracy and stability of the classification results. In the specific text classification process, the model needs to learn to recognize the same semantics under different expressions. The diversified samples generated by the enhancement technology enable the model to be exposed to a wider range of language variants, thereby enhancing its ability to understand different expressions. In addition, data augmentation can effectively alleviate the problem of uneven sample distribution. In many text classification tasks, the number of samples in different categories may be significantly different. By performing targeted enhancement on the samples of minority categories, the sample distribution of each category can be balanced, thereby improving the model's recognition ability for minority categories and improving the overall classification accuracy.

[0016] Based on the above content, an overview of the data enhancement process and implementation applications for text classification tasks are introduced. Next, the specific domain knowledge fusion data enhancement method for text classification in this application is introduced.

[0017] Examples, such as Figure 1 As shown, the present application provides a domain knowledge fusion data enhancement method for text classification, the method comprising:

[0018] Connect to the target domain document library, extract from the target domain document library according to the text classification task, and build a basic corpus.

[0019] Specifically, first, a data interface is established with the selected document library, which usually involves the use of API (application programming interface) or web crawler technology. Access to existing document resources in the target field is achieved by establishing a data interface connected to the target domain document library, which contains a wide range of professional documents, covering various research papers, case studies, review articles, etc. in the target field. The target domain document library is a collection of documents specifically established for a specific field (such as medicine, law, or engineering, etc.), and through this library, it is intended to extract content directly related to the text classification task. The document structure in the target domain document library is analyzed according to specific text classification tasks, such as topic identification. This analysis process includes identifying key parts of the document, such as abstracts, keywords, paper titles, research results, and discussions, so as to accurately locate text paragraphs related to the classification task. For example, in an analysis task for medical literature, special attention will be paid to the research results section to extract descriptions about the effects of drugs. Next, corpus positive samples and corpus negative samples are extracted from these structured parsed contents. Corpus positive samples refer to text paragraphs that are closely related to the text classification task. These samples directly correspond to the target category of the classification task; while corpus negative samples are usually texts that are close to the context position of the positive sample in the same document but do not belong to the target classification in terms of content. This application takes the text classification task of scientific value sentences in the biological field as an example, and uses manual annotation and generative data enhancement methods based on large models to construct a corpus. In order to balance the training data set, reduce model bias, and improve the effectiveness of evaluation indicators, the ratio of positive and negative samples of the constructed corpus is set to 1:1. Specifically, the corpus construction process is to first select sentences describing scientific value from the full text of 1213 scientific and technological documents by manual annotation as positive samples of the training set, totaling 551, and select non-research problem sentences from the context near the sentence as negative samples of the training set, totaling 551. Then, the annotated artificial value sentences and prompt words are used to jointly construct the Prompt required for the large language model to generate, that is, the basic corpus. Ultimately, these extracted positive and negative samples are combined to form a basic corpus that will serve as the primary input for training the data augmentation model, providing rich and targeted training materials for subsequent data augmentation and model training. In this way, high-quality, domain-related training data is obtained at the initial stage of data augmentation, providing a solid foundation for efficient and accurate text classification.

[0020] Data enhancement prompt words are defined according to the target text classification task, and the data enhancement prompt words include task description, seed sentence and generation requirements.

[0021] The task description is used to define the goal of generative enhancement in the target domain, the seed sentence is an example of the text classification task in the target domain, and the generation requirements include the style, grammar and semantic relevance of the generated text.

[0022] Optionally, defining data augmentation prompts involves creating specified text for guiding the data augmentation large language model generation task. This process includes three main components: task description, seed sentences, and generation requirements. Each part is designed for a specific goal to ensure that the generated data can effectively support the needs of the text classification task. The task description is set to clarify the goal of generative augmentation in the target domain. It summarizes the overall task purpose of data augmentation, that is, to generate semantically related but different sentences for each seed sentence to enrich the dataset. This part tells the model what specific tasks need to be completed. For example, in a task for medical research literature classification, the task description may be "generate descriptive text that can reflect the effects of different drug treatments", which helps the model understand that the core purpose of its generation task is to highlight the effects of drugs. The seed sentence is an example sentence of the text classification task in the target domain, which plays the role of providing context and style benchmarks for the data augmentation model. The seed sentences should be carefully selected from high-quality text in the target domain to ensure that they can represent typical language usage and professional knowledge expressions in the domain. For example, if the target domain is environmental science, the seed sentence may be "This study demonstrates the potential of new solar cells in reducing carbon emissions", which provides professional context for scientific research and environmental protection technology. The generation requirements specify the style, grammar, and semantic relevance standards that the generated text should follow. These requirements ensure that the generated text not only meets professional standards in form, but also is highly relevant to the seed sentence in content and is innovative. The generation requirements guide the large language model to express innovatively while maintaining the original semantics. For example, the requirements may include "using a formal scientific paper style, maintaining grammatical precision, and introducing scientific concepts that are different but related to the original seed sentence." The data augmentation prompts obtained by the above steps can guide the large language model to effectively generate text that meets the needs of professional fields and is highly original and adaptable, thereby supporting complex text classification tasks. The implementation of this method not only improves the performance of the classification model, but also enhances the model's understanding and processing capabilities of text in specific fields.

[0023] The data enhancement prompt words and the basic corpus are input into a data enhancement large language model for data enhancement to generate an enhanced corpus.

[0024] Exemplarily, the data enhancement process is performed by inputting the defined data enhancement prompt words and the basic corpus into the data enhancement large language model, and this process is intended to generate a richer and more diverse enhanced corpus to support more efficient text classification tasks. First, the combination of data enhancement prompt words and the basic corpus provides a comprehensive input set, which includes specific task instructions and actual text examples. The task description, seed sentence and generation requirements included in the data enhancement prompt words are all used to guide the model on how to generate and modify text. For example, the task description may specify that a text describing the effect of drug treatment is to be generated, the seed sentence provides a specific text sample, and the generation requirements detail the specific language style and semantic quality that the generated text should follow. Then, these prompt words and the text samples extracted from the basic corpus are input into the data enhancement large language model, which is an advanced machine learning model, usually based on deep learning techniques, such as transformer networks (Transformers), which can understand and generate human language. In this step, the model uses the input prompt words as a guide to generate new text by learning the language patterns and contextual information in the basic corpus. In this process, the model not only repeats the original text, but also creatively generates new text to meet the style and semantic relevance standards specified in the generation requirements, which may include variants, extensions, or re-expressions of the seed sentence to generate diverse text outputs, which are designed to expand the original corpus and increase its diversity. For example, if the seed sentence describes the therapeutic effect of a drug, the enhanced model may generate multiple sentences with different expressions, which express the same concept from different perspectives or in different languages, thereby enriching the content and expression of the corpus. Finally, the generated enhanced corpus contains texts in the original corpus and newly generated texts, which are more diverse and creative, and help improve the performance and accuracy of subsequent text classification tasks. This enhanced corpus can be used directly to train text classification models, or as a basic dataset for further research and analysis. In this way, the data-enhanced large language model is effectively used to combine theory and practice, which not only improves the adaptability of the model to texts in specific fields, but also significantly improves the quality of the dataset and the training effect of the model.

[0025] An instruction data set is constructed, and based on the instruction data set, a low-rank adaptive technology is used to perform fusion feedback learning on the data-enhanced large language model to obtain an optimized data-enhanced large language model.

[0026] Furthermore, constructing the instruction dataset involves selecting high-quality text samples from the enhanced corpus and formulating specific learning tasks for each sample. The instruction dataset includes task description, input data, and expected output data. The task description defines the goals and requirements of model training, for example, generating therapeutic effect description text related to a specific medical condition. The input data is the selected high-quality text sample, and the output data is the text expected to be generated by the model, usually an improved or changed version, showing different expressions or deeper information. For example, in order to further improve the quality of the generated scientific value sentences, the instruction dataset is constructed for model fine-tuning by combining the manually annotated corpus and the selected high-quality generated corpus. Specifically, continuing to take the biological field as an example, 551 manually annotated scientific value sentences are used as seed sentences to generate 1134 scientific value sentences, and then 623 high-quality generated scientific value sentences are obtained from them by manual screening. These seed sentences cover the theoretical importance of the research problem, the innovation of the method, and the applicability of the results. The instruction fine-tuning dataset is constructed based on 1174 high-quality scientific value sentences. The instruction fine-tuning dataset consists of three parts: task description, input, and output. The task description clearly states what the goal of the generation task is, what kind of tasks the model needs to complete, what specific requirements and restrictions are, etc., providing clear guidance for subsequent input and output. The input provides contextual information or prompts for the model to generate. The input data matches the task description and provides the necessary background information for the model to complete the task. The output is the target text, image and other output content generated by the model based on the task description and input. The output must meet the requirements in the task description and achieve the expected effect. Next, the data-enhanced large language model is trained and optimized using low-rank adaptive technology (LoRA). The LoRA technology introduces a low-rank matrix in each network layer of the model to adjust and optimize the weight distribution between layers, which helps the model adapt to specific task requirements more accurately without adding too much computational burden. In the implementation process, the weights of each network layer are optimized with low-rank perturbations, which is achieved through low-rank projection matrices that capture and enhance signals that are particularly important for specific tasks. For example, if the task is to generate a description of the therapeutic effect of a specific disease, some layers of the model may need to be more responsive to terms or sentences related to the mechanism of action of the drug. In this case, LoRA can adjust the weights of these layers so that they are more inclined to generate statements related to the drug's mechanism of action. During the optimization process, by comparing the model output before and after training with the expected output, the low-rank projection matrix is ​​continuously adjusted until the model output meets the preset quality standards. This approach not only enhances the model's adaptability to specific tasks, but also improves the accuracy and relevance of the generated text.Ultimately, the data-enhanced large language model optimized by this low-rank adaptive technology can more effectively process and generate text that is closely related to the target task, greatly improving the quality of the corpus and the practicality of the model. This optimized model provides more accurate and diverse training data for subsequent text classification tasks, thereby achieving higher performance and better results in practical applications.

[0027] According to the optimized data enhancement large language model, an optimized enhanced corpus is obtained, and according to the optimized enhanced corpus, a classification result of the text classification task is obtained.

[0028] Specifically, the process of generating an enhanced corpus through an optimized data augmentation large language model involves applying a model that was previously fine-tuned using low-rank adaptation (LoRA) to the base corpus. The model has been adjusted for a specific task to make it more suitable for generating task-related text. For example, if the model is trained to generate medical text, then at this stage, the model will be able to produce detailed text describing different diseases and treatment effects, which not only have the same language style as the original text, but also have richer and more diverse content. Next, the generated enhanced corpus contains the original text data and the newly generated text by the model. This library now has depth and breadth that the original corpus does not have, providing a richer training foundation for text classification tasks. The text in the enhanced corpus has been optimized by the model to better meet the needs of data diversity and complexity in practical applications, such as in legal text classification, it may include more variant descriptions of specific legal terms, which improves the accuracy of automatic classification of legal documents. Finally, this optimized enhanced corpus is used to obtain classification results for the text classification task. In this phase, classification algorithms suitable for specific classification tasks, such as support vector machines (SVMs), deep neural networks, etc., are used to train classification models based on the data in the enhanced corpus. For example, in a classification task that aims to distinguish the types of research in scientific research articles, the model will learn how to judge whether it belongs to theoretical research, experimental research, or review types based on the language used and content in the article. During the training process, the algorithm will learn the use of various expressions and terms from the enhanced corpus, so that it can perform classification tasks more accurately. Through the above process, not only the accuracy of text classification is significantly improved, but also the generalization ability and practicality of the model are enhanced by generating richer and more relevant training data.

[0029] Furthermore, if Figure 2 As shown, construct an instruction data set, including:

[0030] A quality screening module is constructed, wherein the quality evaluation indicators of the quality screening module include an importance indicator, an innovation indicator, and an applicability indicator that characterize the corpus; a corpus evaluation is performed on the enhanced corpus according to the quality screening module, and a screening corpus with a quality greater than or equal to a preset quality is screened; based on the screening corpus, the instruction data set is constructed, and the instruction data set includes a task description, and input data and output data corresponding to the task description.

[0031] Furthermore, the purpose of constructing the instruction dataset is to ensure that the training data used by the data augmentation model meets the highest quality standards. This process involves three main steps: constructing a quality screening module, evaluating and screening the augmented corpus, and constructing an instruction dataset based on the screening results. First, the quality screening module is constructed to systematically evaluate the quality of each text data in the augmented corpus. This module sets several key quality evaluation indicators, including the importance index, innovation index, and application index of the characterization corpus. Among them, the importance index of the characterization corpus is used to evaluate the centrality or criticality of the text content in its field. For example, in medical literature, the description of a specific disease or treatment method may be of great importance; the innovation index is used to evaluate the novelty of the text content relative to known information. For example, an article proposing a new drug synthesis method will score higher on this indicator; and the application index is used to evaluate the practicality and practical application potential of the text content, such as a detailed description of the new technology application scheme in a technical document.

[0032] Next, the enhanced corpus is evaluated using the quality screening module. This step involves comprehensively scoring each piece of data in the library, screening out qualified corpora according to preset quality standards, and ensuring that only the highest-quality texts are selected into the final training set, thereby improving the learning efficiency of the data enhancement model and the quality of the final output. For example, the screening process may exclude texts that score low on the innovation index and retain texts with rich content and unique insights. Finally, an instruction dataset is constructed based on the screened corpus, which not only includes texts selected from the screened corpus as input data, but also designs corresponding output data that reflects the expected model response. Each input data is accompanied by a clear task description to guide the model on how to process the data. The task description may be specific to how to rewrite a sentence to increase its persuasiveness, or how to reorganize information to improve the logic of the text. For example, if the input data is a description of the treatment of a certain disease, the corresponding output data may be to require the model to generate a more detailed and more comprehensive version of the treatment plan. This approach ensures that every piece of data used to train the data augmentation model is carefully screened and optimized, thereby maximizing the training effect and the practical application value of the model.

[0033] Furthermore, based on the instruction data set, low-rank adaptive technology is used to perform fusion feedback learning on the data-enhanced large language model, and the method includes:

[0034] Analyze multiple network layers of the data enhanced large language model; introduce a low-rank projection matrix group on each of the multiple network layers, and obtain multiple low-rank projection matrix groups corresponding to the multiple network layers, wherein each low-rank projection matrix group includes a first projection matrix and a second projection matrix; perform low-rank perturbation optimization on the weights of the multiple network layers according to the multiple low-rank projection matrix groups until the optimization target is met, and obtain the optimized data enhanced large language model.

[0035] Specifically, the data-augmented large language model used is structurally analyzed to determine the functions and characteristics of each network layer, including identifying the specific tasks undertaken by each layer (such as feature extraction, semantic analysis, etc.) and their role in data processing. For example, the primary layer may be mainly responsible for processing basic language structures, while the higher-level layers process complex semantic associations. Subsequently, a set of low-rank projection matrices, including the first and second projection matrices, are introduced on each parsed network layer. These matrices are intended to redefine and optimize the information transmission path between layers so that the model can more sensitively adjust its response to adapt to specific task requirements. Among them, the first projection matrix is ​​responsible for capturing the key aspects of the input features, while the second projection matrix adjusts how these features affect the processing of subsequent layers. Next, the weights of each network layer are optimized using the low-rank projection matrix group. This process iteratively adjusts the values ​​in the matrix to minimize the objective function, usually the prediction error or classification error. The optimization process continues until the model output reaches a preset performance standard or optimization goal. For example, in a task for automatic classification of legal documents, the optimization may focus on improving the model's ability to parse the relationship between different legal concepts. Through the above steps, low-rank adaptive technology enables data-enhanced large language models to optimize specific tasks more accurately while maintaining their original complexity and depth. This method not only improves the operational efficiency and output quality of the model, but also enables it to show better adaptability and accuracy in practical applications. In addition, due to the use of low-rank matrices, the computing resources required during the training and adjustment of the model can be effectively controlled, further enhancing the practicality of the model.

[0036] Furthermore, low-rank perturbation optimization is performed on the weights of the multiple network layers respectively according to the multiple low-rank projection matrix groups, wherein the expression for low-rank perturbation optimization of the weights of any network layer includes:

[0037] h i ′=f i (x i )+A i B i fi (x i ) = f i (x i )+Δ i ; Among them, h i ′ is the corrected forward calculation formula of the ith network layer, h i =f i (x i ), f i (x i ) is the original forward calculation formula of the i-th network layer, A i is the first projection matrix of the i-th network layer, B i is the second projection matrix of the i-th network layer, Δ i =A i B i f i (x i ) represents the introduced low-rank perturbation correction term.

[0038] Optionally, a specific increment is introduced at each network layer of the model to improve the overall performance of the model. This approach is particularly suitable for large language models with data augmentation that require fine-grained tuning. Specifically, h i ′=f i (x i )+A i B i f i (x i ) = f i (x i )+Δ i Describes how to output h of each layer i Here, f i (x i ) represents the output of the i-th layer of the original model, A i and B i are two low-rank projection matrices specially designed for this layer. These two matrices are transformed by f i (x i ), which is the original output of the layer, generates an increment Δ i This increment is a modification of the original output that is used to fine-tune the behavior of the model. Specifically, A i and B i The design of is to capture and enhance the most critical information of this layer for a specific task. Next, by applying these low-rank projection matrices, the increment Δ i is calculated and added to the original output f i (x i ) to form the adjusted output h i'. The purpose of this process is to enhance the model's responsiveness to specific types of inputs through subtle adjustments without significantly increasing computational complexity. In this way, each layer of the entire model is gradually optimized so that the final model output is more in line with specific application requirements, such as more accurate recognition and classification of complex text data. The implementation of this method not only improves the accuracy of the model, but also enhances the model's ability to handle specific tasks, thereby achieving better performance in practical applications.

[0039] Furthermore, the optimization objective is constructed by minimizing a loss function, wherein the expression of the loss function includes:

[0040] Among them, θ is the fixed parameter of the data-enhanced large language model before optimization, represents the multiple low-rank projection matrix groups introduced, L is the total number of multiple low-rank projection matrix groups, is a training dataset based on the instruction dataset, (x,y)∈D represents a single sample x and the corresponding label y in the training dataset, l is the cross entropy loss based on the text classification task, y is the true label, which is used to measure the error between the predicted value and the true label, It is a prediction function optimized based on a data-augmented large language model and a low-rank projection matrix group.

[0041] Furthermore, the performance of the data-augmented large language model is further improved by introducing a specific optimization strategy. This strategy mainly defines a loss function for the parameters θ in the model and the low-rank projection matrix A of each layer. i and B i Optimize. The loss function is expressed as The loss function is calculated based on the entire dataset Specifically, this function accumulates the prediction errors for each pair of samples in the dataset, where represents the predicted output of the optimized model for input x, and y is the corresponding true label. This method evaluates the performance of the model by comparing the difference between the predicted output and the true output, and the goal is to minimize these differences. The goal of optimization is to adjust the model parameters θ and the inter-layer projection matrix A i and B i , to ensure that the entire model performs best on a specific task. Each low-rank projection matrix A i and B i Responsible for fine-tuning the model's internal representation to more effectively handle specific types of data features. By adjusting these matrices, the model can more accurately adapt to complex patterns and changes in the data. The optimization process involves Iteratively adjust θ and In each iteration, the loss function value under the current model parameters is calculated, and the parameters are updated through gradient descent or other optimization algorithms to gradually reduce the prediction error. This includes A i and B i Adjustments are made to ensure that these matrices can effectively adjust the output of the layer to suit the specific needs of the training data. In this way, the model can not only more accurately identify the emotional tendencies in the text, but also better understand the subtle differences in emotional expressions. Through the above methods, the performance of the data-augmented large language model can be effectively improved, enabling it to provide more accurate and reliable output in specific applications.

[0042] Furthermore, extracting from the target domain document library according to the text classification task to construct a basic corpus includes:

[0043] Perform structured analysis on each document in the target domain document library and output structured analysis content; extract the structured analysis content according to the text classification task to obtain corpus positive samples and corpus negative samples, wherein the corpus positive samples are a corpus set matching the text classification task, and the corpus negative samples are a corpus set corresponding to the context of the corpus positive samples; construct a basic corpus based on the corpus positive samples and the corpus negative samples.

[0044] Exemplarily, each document in the target domain document library is structured parsed to automatically extract structured information from a large number of documents. The purpose of structured parsing is to convert unstructured text in the document (such as the body of the article) into easy-to-process structured data (such as title, abstract, keywords, section titles and content). For example, natural language processing tools and parsing algorithms are used to identify and mark different parts of the article, such as introduction, methods, results and discussion. After completing the structured parsing of the document, the corpus positive samples and corpus negative samples are extracted from the parsed content according to specific text classification requirements. Corpus positive samples refer to those parts that are directly related to the text classification task. For example, in a task aimed at classifying the effects of medical treatments, positive samples may include paragraphs describing specific treatment methods and treatment results. In contrast, corpus negative samples include texts that are adjacent to positive samples in content but do not directly reflect the classification target, such as pathological discussions that are not directly related to treatment. Then, the corpus positive samples and negative samples are used to build a basic corpus. This library will serve as the basis for data enhancement and machine learning model training to support more complex data processing and analysis tasks. The construction process includes integrating these samples, formatting the data to meet the requirements of the training model, and performing necessary data cleaning and preprocessing, such as removing noise, standardizing text formats, etc. For example, if the task is to identify and classify literature about specific drug treatments, positive samples may contain key information such as drug names, usage methods, clinical trial results, etc., while negative samples may include disease background introductions or discussions of non-target effects of drugs. In this way, the basic corpus will contain a wide range of relevant and non-relevant texts, providing a solid foundation for in-depth learning and precise classification. In this way, not only the efficiency and accuracy of data processing are improved, but also the text classification model is ensured to perform well in challenging practical applications. This systematic approach provides reliable technical support for the efficient processing and analysis of large-scale literature data.

[0045] Furthermore, the classification result of the text classification task is obtained according to the optimized enhanced corpus, and the method further includes:

[0046] Corpus fusion is performed based on the optimized enhanced corpus and the basic corpus to obtain a fused corpus; and classification results of the text classification task are obtained based on the fused corpus.

[0047] Specifically, the optimized enhanced corpus is fused with the original base corpus. The optimized enhanced corpus contains texts generated by the data augmented large language model, which have been specifically adjusted and optimized to better reflect the needs of specific text classification tasks. The base corpus contains original texts directly extracted from the target domain literature. The purpose of fusing the two corpora is to combine the breadth of the original texts with the specificity of the enhanced texts to create a more comprehensive and detailed training and testing dataset. Based on the fused corpus, a suitable machine learning or deep learning model is used to perform the text classification task. This process includes selecting a suitable classification algorithm (such as support vector machine, random forest or neural network) to train the model and finally evaluate its performance on the classification task. During the training process, the model learns how to identify and distinguish different categories from the text features in the fused corpus. After training, the model is able to effectively classify new input text and provide decision support about which category it belongs to. For example, in a task aimed at classifying legal documents, the fused corpus allows the model to learn not only the basic usage of legal terms, but also understand the specific application of these terms to different legal cases. Through the above process, the fused corpus composed of the optimized enhanced corpus and the basic corpus can be effectively utilized to improve the accuracy and efficiency of text classification tasks. This method not only enhances the model's ability to understand specific text types, but also improves its generalization ability in practical applications.

[0048] Through the technical scheme of the above embodiment, the domain knowledge fusion data enhancement method for text classification provided by the present application ensures the domain relevance of the enhanced data by extracting representative positive and negative samples from the professional literature of the target field as the basic data set; by analyzing the metadata elements in the text classification task, constructing the data enhancement prompt words that integrate the domain knowledge, and guiding the large model to generate grammatically correct, semantically coherent, and rich in content High-quality enhanced samples. At the same time, the generated diversified and innovative sentences enhance the richness and coverage of the data, and improve the generalization ability and adaptability of the model in the text classification task. In addition, fine-tuning is also performed on the corpus of a specific field or style, and enhanced sentences that meet the target field or style are generated, which enhances the quality and availability of the generated data. It solves the technical problems that the existing technology cannot effectively combine domain knowledge for deep text data enhancement, the richness and coverage of the data are poor, resulting in insufficient semantic depth and domain adaptability of the generated samples, and the quality and availability of the generated data are limited, and achieves the technical effect of effectively generating high-quality, domain-adaptable text samples, improving the accuracy and generalization ability of the text classification model, and enhancing the availability of the generated data.

[0049] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0050] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A domain knowledge fusion data enhancement method for text classification, characterized in that: The method comprises: Connecting to the target domain document library, extracting from the target domain document library according to the text classification task, and constructing a basic corpus; Define data enhancement prompt words according to the target text classification task, wherein the data enhancement prompt words include task description, seed sentence and generation requirements; Inputting the data enhancement prompt words and the basic corpus into a data enhancement large language model to perform data enhancement and generate an enhanced corpus; Constructing an instruction data set, and performing fusion feedback learning on the data-enhanced large language model using a low-rank adaptive technology based on the instruction data set to obtain an optimized data-enhanced large language model; According to the optimized data enhancement large language model, an optimized enhanced corpus is obtained, and according to the optimized enhanced corpus, a classification result of the text classification task is obtained.

2. The domain knowledge fusion data enhancement method for text classification according to claim 1, characterized in that: Build an instruction dataset, including: Constructing a quality screening module, wherein the quality assessment indicators of the quality screening module include an importance indicator, an innovation indicator, and an applicability indicator that characterize the corpus; Performing corpus evaluation on the enhanced corpus according to the quality screening module, and screening a screening corpus with a quality greater than or equal to a preset quality; The instruction data set is constructed according to the screening corpus, and the instruction data set includes a task description, and input data and output data corresponding to the task description.

3. The domain knowledge fusion data enhancement method for text classification according to claim 1, characterized in that: Based on the instruction data set, low-rank adaptive technology is used to perform fusion feedback learning on the data-enhanced large language model, and the method includes: Parsing multiple network layers of the data-augmented large language model; Introducing a low-rank projection matrix group on each of the multiple network layers, and obtaining multiple low-rank projection matrix groups corresponding to the multiple network layers, wherein each low-rank projection matrix group includes a first projection matrix and a second projection matrix; Low-rank perturbation optimization is performed on the weights of the multiple network layers according to the multiple low-rank projection matrix groups until the optimization target is met, thereby obtaining an optimized data-enhanced large language model.

4. The domain knowledge fusion data enhancement method for text classification according to claim 3, characterized in that: The weights of the multiple network layers are respectively subjected to low-rank perturbation optimization according to the multiple low-rank projection matrix groups, wherein the expression for performing low-rank perturbation optimization on the weights of any network layer includes: h i ′=f i (x i )+A i B i f i (x i )=f i (x i )+Δ i ; Among them, h i ′ is the corrected forward calculation formula of the ith network layer, h i =f i (x i ), f i (x i ) is the original forward calculation formula of the i-th network layer, A i is the first projection matrix of the i-th network layer, B i is the second projection matrix of the i-th network layer, Δ i =A i B i f i (x i ) represents the introduced low-rank perturbation correction term.

5. The domain knowledge fusion data enhancement method for text classification according to claim 4, characterized in that: The optimization objective is constructed by minimizing a loss function, wherein the expression of the loss function includes: Among them, θ is the fixed parameter of the data-enhanced large language model before optimization, represents the multiple low-rank projection matrix groups introduced, L is the total number of multiple low-rank projection matrix groups, is a training dataset based on the instruction dataset, (x,y)∈D represents a single sample x and corresponding label y in the training dataset, is the cross entropy loss based on the text classification task, y is the true label, which is used to measure the error between the predicted value and the true label. It is a prediction function optimized based on a data-augmented large language model and a low-rank projection matrix group.

6. The domain knowledge fusion data enhancement method for text classification according to claim 1, characterized in that: The data enhancement prompt words include task description, seed sentence and generation requirements; The task description is used to define the goal of generative enhancement in the target domain, the seed sentence is an example of the text classification task in the target domain, and the generation requirements include the style, grammar and semantic relevance of the generated text.

7. The domain knowledge fusion data enhancement method for text classification according to claim 1, characterized in that: Extracting from the target domain document library according to the text classification task to construct a basic corpus includes: Performing structured analysis on each document in the target domain document library and outputting structured analysis content; Extracting the structured parsing content according to the text classification task to obtain a corpus positive sample and a corpus negative sample, wherein the corpus positive sample is a corpus set matching the text classification task, and the corpus negative sample is a corpus set corresponding to the context of the corpus positive sample; A basic corpus is constructed according to the corpus positive samples and the corpus negative samples.

8. The domain knowledge fusion data enhancement method for text classification according to claim 1, characterized in that: Obtaining the classification result of the text classification task according to the optimized enhanced corpus, the method further includes: Performing corpus fusion according to the optimized enhanced corpus and the basic corpus to obtain a fused corpus; According to the fused corpus, a classification result of the text classification task is obtained.

Images