Intelligent text data label generation method and system

Abstract

The invention provides an intelligent text data label generation method and system, belongs to the technical field of intelligent text data label generation, and particularly provides an intelligent text label generation method by introducing few-sample self-training, multi-modal fusion and interpretable AI technologies. According to the method, labels can be effectively generated under the condition of data scarcity, richer and context-related labels can be generated in combination with multi-modal information, and meanwhile, the trust and understanding of users on the method are enhanced by providing a transparent label generation interpretation process.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent text data label generation, and in particular, relates to an intelligent text data label generation method and system. Background Art

[0002] With the advent of the big data era, massive amounts of text data are rapidly accumulating. How to effectively and automatically analyze and annotate these data has become a crucial technical issue. The text tag generation method is widely used in search engines, recommendation methods, natural language processing and other fields. It can generate brief tags suitable for the text through semantic analysis of text data, thereby improving the retrieval, classification and understanding capabilities of text data. Traditional text tag generation technology mainly relies on two methods: one is a tag generation method based on artificial rules, and the other is an automatic tag generation method based on machine learning and deep learning models.

[0003] The typical approach of the manual rule method is to generate labels for texts by having experts design a series of rules and keyword matching models based on domain knowledge. However, this method relies heavily on the experience of domain experts, has poor applicability and low scalability, and cannot be used flexibly in different fields. Especially today when the amount of data is increasing rapidly, relying on manual rules is both time-consuming and labor-intensive, and cannot meet the needs of high-frequency updates. In addition, rule methods are often only applicable to specific scenarios and cannot effectively handle complex semantics, polysemous words, or semantic changes in different contexts, resulting in the generated labels being difficult to accurately reflect the complex semantics of the text content.

[0004] The automatic label generation method based on machine learning and deep learning mainly relies on the training of large-scale data sets. By learning from massive annotated data, the model can extract the deep semantics of the text and generate corresponding labels. The emergence of deep learning technologies such as BERT, GPT and other language models has greatly improved the accuracy and efficiency of label generation. However, existing deep learning methods also face some limitations. First, the training process of such models usually relies on a large amount of annotated data, while in many emerging fields or fields with less data, the lack of annotated data makes it difficult to fully train the model. Secondly, most existing models are single-modal models, which mainly rely on text input for label generation, and it is difficult to combine other modal data (such as images, audio, etc.) to generate more comprehensive labels. Especially in scenarios with rich multimodal information such as social media and news, the label generation of the present invention that relies solely on text is difficult to capture all information. In addition, most of the current deep learning models are black box models, lacking transparency, and users cannot clearly understand the basis for the model to generate labels, especially in sensitive fields such as medical, finance, and law. The unexplainable label generation process may cause users to lose trust in the present invention. Therefore, although the existing technology can improve the level of automation of text label generation to a certain extent, it still has major shortcomings in terms of poor cross-domain adaptability, low transparency of label generation, and insufficient ability to integrate multimodal information. Summary of the invention

[0005] The purpose of the present invention is to design an intelligent text data label generation method and system, and propose an intelligent text label generation method and the present invention by introducing few-sample self-training, multimodal fusion and explainable AI technology. The present invention can not only effectively generate labels when data is scarce, but also combine multimodal information to generate richer and more context-related labels. At the same time, by providing a transparent label generation and interpretation process, it enhances the user's trust and understanding of the present invention.

[0006] In order to achieve the above object, a first aspect of the present invention provides a method for generating intelligent text data tags, the method comprising the following steps: Step S1, obtaining a text data set extracted from a small amount of annotated samples, establishing a label candidate model based on a few samples and a sparse data loss function according to the obtained text data set, and introducing a multi-level label generation mechanism in the label candidate model, wherein the label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model; Step S2: input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution, and design a pseudo label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo labels for the screened samples, and then combine the generated pseudo labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model; The pseudo-label selection strategy designed for unlabeled data is used to screen unlabeled samples, and the specific steps include: Design a confidence threshold and compare it with each unlabeled sample. When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label. Otherwise, the sample is skipped to avoid introducing erroneous labels with low confidence. If the highest probability predicted by the label candidate model is less than the confidence threshold, the sample will not generate a pseudo label. Step S3: Map the multimodal data in the unlabeled dataset through a shared embedding layer, and use a weighted fusion mechanism to dynamically adjust the weights of each modality according to its importance in a given task, and then perform weighted fusion to obtain a unified multimodal representation. Step S4: Generate label distribution using a dynamic label generation network according to the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model; The adaptive mechanism of label generation includes a modality uncertainty weight mechanism and an inter-modality consistency constraint; the modality uncertainty weight mechanism calculates the prediction uncertainty of each modality and takes uncertainty as an important component of the label weight in the label generation process; the inter-modality consistency constraint is used to constrain the differences between label distributions generated by different modalities; Step S5: Make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model adjusted by user feedback.

[0007] Furthermore, in step S1, in order to process the extracted text dataset, it is necessary to map the text into a high-dimensional vector space through a pre-trained language model; The loss function based on sparse data is designed as follows: First, define a label candidate model , used to embed Generate preliminary label candidates, as follows: , in, is the initial label candidate generated, represents the number of potential labels; Design a regularized loss function to deal with the few-sample problem, as follows: , in, is the cross entropy loss, which measures the difference between the true label and the predicted label. is the regularization coefficient, is the regularization term; The multi-level label generation mechanism is shown as follows: Each text embedding By labeling candidate models Generate preliminary label probability distribution , by introducing a multi-class label activation mechanism to generate a preliminary label candidate set to enhance label diversity, as shown below: , in, Each element in represents the probability of generating a certain label; At the same time, in order to deal with semantic ambiguity and uncertainty, the selection mechanism is defined as: , That is, choose the first The label with the highest probability is taken as the preliminary label candidate.

[0008] Furthermore, the regularization term According to the adaptive adjustment of the confidence of each feature, the largest part of the discrete features is penalized to avoid overfitting of small sample data in high-dimensional space, which is expressed as follows: , in, Indicates The model weights of feature dimensions; is embedded The Dimensional features; and They are the embedded features in The mean and standard deviation of the dimension; Among them, when Closest When , the regularization effect is the strongest, indicating that this feature dimension is the most uncertain.

[0009] Furthermore, the step S2 further includes: A self-training loss function is designed according to a new training set to train the label candidate model. The new data set includes annotated data and pseudo-label data with high confidence. At the same time, an additional penalty term is added to the pseudo-label data to reduce the noise that may be caused by the pseudo-label. The self-training loss function is expressed as follows: , in, To mark the number of samples, is the number of pseudo labels generated; is the cross entropy loss, which measures the difference between the true label and the predicted label; It is a newly introduced pseudo-label uncertainty penalty term, which aims to weight the pseudo-label according to its confidence level to prevent pseudo-labels with low confidence from adversely affecting model training; and are important coefficients used to balance the labeled data and pseudo-label data; the pseudo-label uncertainty penalty term It is expressed as follows: , in, Represents unlabeled samples For The predicted probability of each category; In the process of label candidate model, an uncertainty calibration is performed after each iteration. The uncertainty calibration adjusts the weight of the pseudo label for model training according to the prediction confidence of the pseudo label and the stability of the model to the label. And the entropy value of the label Together they determine the weight of the pseudo-label in the loss function, expressed as follows: , in, is the adjustment parameter, is a pseudo-label sample The weight in the training loss. Samples with higher confidence and lower entropy will have higher weights, and vice versa; The optimized self-training loss function is expressed as follows: .

[0010] Furthermore, the weighted fusion mechanism comprises the following specific steps: For each modality, calculate its weight for the current task, for the sample , modal Weight The feature vector of each modality and the shared weight vector The inner product is determined as follows: , in, is the mapping of the modality m in the embedding layer; the weighted fusion mechanism ensures that the text ,image , Audio Ability to adaptively adjust weights based on their relevance to the task.

[0011] Furthermore, the generating label distribution using a dynamic label generation network according to the fused unified semantic representation specifically includes: Define a multimodal label generation function , whose input is a multimodal unified representation , the output is the label probability distribution for this sample , the distribution is generated by the Softmax function and is expressed as follows: , in, , the Softmax function converts the generated results into a probability distribution, indicating the possibility of each label; Based on the label distribution of each sample A dynamic Top-K tag selection mechanism is designed to ensure that the generated tags can be adaptively selected according to the multimodal features of the current input. The dynamic tag selection process of the dynamic Top-K tag selection mechanism selects the top-K tags with the highest confidence. This is achieved by using a tag, as shown below: , in The size can be adjusted dynamically.

[0012] Furthermore, in step S4, uncertainty is introduced into the label generation process to ensure that the modality with high uncertainty contributes less to the final generated label, while the modality with high confidence plays a greater role. The uncertainty is based on uncertainty calculation, which uses the output probability distribution of each modality To estimate the uncertainty, for each mode Representation , whose uncertainty is calculated by entropy and is expressed as follows: , in, Representing modality For Category The predicted probability of , C represents the total number of categories; Design multi-modal adaptive weights and dynamically adjust the trust of the label candidate model on each modality by adjusting the weight of the modality, as shown below: , in, It is a balance parameter used to control uncertainty. When the confidence of a certain mode is the highest, its weight If it is closest to 1, the label candidate model will trust the prediction of this modality the most; otherwise, the influence of this modality will be reduced; The inter-modality consistency constraint is calculated based on the modality consistency loss, which is expressed as follows: , in, represents the consistency loss, Representing modality The output probability distribution of Representing modality The output probability distribution of .

[0013] Furthermore, the adaptive mechanism for label generation is combined with the prediction error to further optimize the loss function of the optimized label candidate model, which is expressed as follows: , in, is the cross entropy loss of the label, which measures the difference between the generated label and the true label; It is the modality consistency loss, which ensures that the labels generated by different modalities are consistent; It is a hyperparameter that controls the weight of the loss term and is used to adjust the impact of different mechanisms on model optimization.

[0014] Furthermore, the steps of making the labels generated by the label candidate model transparent and explainable include: For each sample , explain the generation process of each label, and show the relationship between the label and the input data according to the mode in the input data and the corresponding weight; At the same time, the Shapley value method is used to analyze the contribution of multimodal features in each label generation process, and then the specific role of each feature in label generation is demonstrated, as shown below: For samples , its multimodal features The contribution of each dimension feature in is calculated by Shapley value, and the formula is: , in, represents the index of the feature, Features For samples The contribution of tag generation.

[0015] In a second aspect of the present invention, a system for generating intelligent text data tags is provided, the system comprising: A few-sample data training module is used to obtain a text dataset extracted from a small number of annotated samples, establish a label candidate model based on a few-sample and sparse data loss function according to the obtained text dataset, and introduce a multi-level label generation mechanism in the label candidate model. The label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model; The label candidate model optimization module is used to input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution, and at the same time, design a pseudo-label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo-labels for the screened samples, and then combine the generated pseudo-labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model; The pseudo-label selection strategy designed for unlabeled data is used to screen unlabeled samples, and the specific steps include: Design a confidence threshold and compare it with each unlabeled sample. When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label. Otherwise, the sample is skipped to avoid introducing erroneous labels with low confidence. If the highest probability predicted by the label candidate model is less than the confidence threshold, the sample will not generate a pseudo label. The data fusion module is used to map the multimodal data in the unlabeled dataset through a shared embedding layer. At the same time, a weighted fusion mechanism is adopted to dynamically adjust the weights of each modality according to its importance in a given task and then perform weighted fusion to obtain a unified multimodal representation after fusion. The label distribution module is used to generate label distribution using a dynamic label generation network based on the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model; The adaptive mechanism of label generation includes a modality uncertainty weight mechanism and an inter-modality consistency constraint; the modality uncertainty weight mechanism calculates the prediction uncertainty of each modality and takes uncertainty as an important component of the label weight in the label generation process; the inter-modality consistency constraint is used to constrain the differences between label distributions generated by different modalities; The label optimization generation module is used to make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model after adjustment based on user feedback.

[0016] The beneficial technical effects of the present invention are at least as follows: First, the present invention solves the problem of existing label generation relying on large-scale annotated data through the self-training technology of a small number of samples. In many emerging fields or long-tail fields, the annotated data is extremely limited. The present invention uses the self-training technology to perform secondary training and iterative optimization of the model after the initial training of the model using the generated high-confidence pseudo-labels, so that the model can still generate high-quality labels with only a small amount of annotated data. This greatly improves the cross-domain adaptability of the present invention, and is particularly suitable for emerging fields or scenarios with scarce data, overcoming the problem of inaccurate label generation due to lack of data in the prior art.

[0017] Secondly, the present invention introduces multimodal fusion technology to solve the limitations of the existing single-modal information label generation in the present invention. Realistic text data is often accompanied by other modal data such as images, audio, and video, and labels generated by text alone often cannot fully reflect the complexity of the information. The present invention uses a multimodal fusion model (such as CLIP, ALIGN, etc.) to uniformly encode text, images, audio and other information, and generates more accurate and context-related labels through multimodal comparative learning. This not only enhances the accuracy of label generation, but also enables the present invention to work in cross-modal data scenarios, effectively making up for the shortcomings of the prior art in multimodal information processing.

[0018] Finally, the present invention introduces an explainable AI module to solve the "black box" problem of the existing label generation of the present invention. In order to enhance the user's understanding and trust in the label generation results, the present invention uses explainable AI technology (such as SHAP, LIME, etc.) to provide clear explanations while generating labels, showing the relationship between generated labels and input data. Users can understand the basis behind label generation, especially in sensitive fields (such as medical, financial, legal, etc.). The transparent label generation process improves the credibility and user experience of the present invention. The interpretability of the present invention not only enhances the transparency of the present invention, but also provides users with a feedback channel. Users can adjust the labels according to the interpretation results. The present invention further optimizes the model through reinforcement learning to improve the accuracy and personalization of the generated results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0020] Figure 1 The present invention is a flowchart of a method for generating intelligent text data tags according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0022] In one or more embodiments, Figure 1 As shown, a method for generating intelligent text data tags is disclosed in the present invention, and the method comprises the following steps: Step S1, obtain a text data set extracted from a small number of annotated samples, establish a label candidate model based on a few samples and a sparse data loss function according to the obtained text data set, and introduce a multi-level label generation mechanism in the label candidate model, the label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model.

[0023] Among them, the input data is a text dataset extracted from a small number of labeled samples, expressed as: , in, represents a text sample, is the true label of the sample, is the number of small number of labeled data samples. This data is scarce, and the few-shot learning method must be used to generate preliminary label candidates to ensure the rationality and scalability of label generation.

[0024] In order to process these text data, the text is first mapped into a high-dimensional vector space through a pre-trained language model (such as a variant of BERT). Indicates Text Semantic embedding of: , in, , This step is the basis of the entire patent process. The pre-trained model can effectively extract semantic information from the text and provide efficient feature representation for few-sample learning.

[0025] Furthermore, due to the limited amount of labeled data, the present invention must effectively train the model under the condition of few samples. In order to avoid overfitting the model to a small amount of data and ensure the quality of generated labels, the present invention designs a domain adaptive regularization term to enable the model to have stronger generalization ability when processing sparse data: First define a model , used to embed Generate preliminary label candidates: , in, is the initial label candidate generated, represents the number of potential labels.

[0026] Next, the present invention designs the following regularization loss function to deal with the few-sample problem: , in, is the cross entropy loss, which measures the difference between the true label and the predicted label. is the regularization coefficient. The key innovation lies in the regularization term The present invention defines: , in, Indicates The model weights of feature dimensions; is embedded The Dimensional features; and They are the embedded features in The regularization term is adaptively adjusted according to the confidence of each feature, penalizing those parts with larger discrete features to avoid overfitting of small sample data in high-dimensional space, thereby enhancing the generalization ability of the model.

[0027] when Close to When , the regularization effect is enhanced, indicating that the feature dimension is relatively uncertain. This innovative regularization design can help the present invention train a more robust model in a few-sample environment, prevent the model from overfitting to a small amount of labeled data, and adapt to the data characteristics of new fields.

[0028] Furthermore, in order to ensure the quality of tag candidate generation, the present invention needs to introduce a multi-level tag generation mechanism to generate high-quality tag candidates by considering the interaction between tags and semantic embedding. Each text embedding Through the mapping function Generate preliminary label probability distribution In order to enhance the diversity of labels, the present invention generates a preliminary label candidate set by introducing a multi-class label activation mechanism: , in, Each element in represents the probability of generating a certain label.

[0029] Furthermore, the present invention hopes to generate Top-K tag candidates. In order to deal with semantic ambiguity and uncertainty, the selection mechanism is defined as: , That is, choose the first The label with the highest probability is used as the preliminary label candidate. Among them, the multi-label selection mechanism allows the present invention to not only generate the most likely label, but also consider multiple possible labels, so as to adapt to the polysemy or label ambiguity that may exist in the text under the condition of few samples.

[0030] Furthermore, after completing the above tag candidate generation process, the present invention will output a preliminary tag candidate set ,in represents the number of samples (including unlabeled data). The candidate label set for each sample is composed of The most likely labels are composed as input for the next step of pseudo label generation and self-training iteration. After optimization based on domain adaptive regularization, it can effectively deal with the few-sample problem, and the output label candidates are of high quality and have strong generalization ability.

[0031] This step is not just a simple few-sample model training. It generates high-quality preliminary label candidates by introducing the sparse regularization design and multi-label selection mechanism in the patent scenario, laying a solid foundation for subsequent pseudo-label generation and self-training iteration.

[0032] Step S2: input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution. At the same time, design a pseudo-label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo-labels for the screened samples, and then combine the generated pseudo-labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model.

[0033] Specifically, from the output of step 1, that is, the initially trained label candidate model , the present invention has obtained a model that can perform preliminary classification of text and generate labels. At this time, the input to step 2 is the unlabeled dataset ,in is the number of unlabeled samples. These unlabeled samples will be used to generate pseudo labels and iteratively optimize the preliminary model through self-training. Through the model Generate a preliminary label probability distribution: , in, is the text embedding calculated in step 1, is the preliminary label distribution predicted by the model, Indicates the number of possible label categories.

[0034] At this point, the core of the problem is how to generate reliable pseudo labels on unlabeled data to optimize the model's capabilities. Directly using predictions from unlabeled data may introduce noise, so this paper introduces a confidence screening mechanism to only retain those predictions with high confidence from the model: The pseudo label selection strategy is to select for each unlabeled sample , the present invention introduces a confidence threshold When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label, otherwise the sample is skipped to avoid introducing low-confidence erroneous labels. The specific formula is as follows: , here, is the generated pseudo label; if the highest predicted probability Less than threshold , then no pseudo label will be generated for this sample. This mechanism ensures that pseudo labels are only generated for samples that the model has high confidence in, reducing the chance of errors propagating into the model.

[0035] Furthermore, the generated pseudo labels can be combined with the original labeled samples to form a larger training set to further optimize the model. Contains annotated data and high-confidence pseudo-labeled data: , in, is the small amount of labeled data set in step 1. Next, the present invention trains the model on this extended data set. The designed self-training loss function not only considers the cross entropy loss of the labeled data, but also needs to add additional penalty terms to the pseudo-labels to reduce the noise that may be caused by the pseudo-labels. The specific self-training loss function is defined as: , in, To mark the number of samples, is the number of pseudo labels generated; is the cross entropy loss, which measures the difference between the true label and the predicted label; It is a newly introduced pseudo-label uncertainty penalty term, which aims to weight the pseudo-label according to its confidence level to prevent pseudo-labels with low confidence from adversely affecting model training. and are important coefficients used to balance the labeled data and pseudo-labeled data respectively.

[0036] Pseudo label uncertainty penalty Designed for: , in, Represents unlabeled samples For The predicted probability of a category. This term is actually the entropy value of the predicted label. The higher the entropy value, the greater the uncertainty of the model about the sample. The present invention uses this entropy value to perform weighted penalty on pseudo-label data to prevent samples with large uncertainty from affecting model training.

[0037] Furthermore, as the iteration proceeds, the quality of the pseudo-labels of the model may gradually improve. In order to further ensure the quality of the pseudo-labels and reduce the risk of noise, the present invention performs an uncertainty calibration after each round of iteration. The uncertainty calibration mechanism adjusts the weight of the pseudo-label for model training according to the prediction confidence of the pseudo-label and the stability of the model for the label.

[0038] Specifically, the confidence of the prediction And the entropy value of the label Together they determine the weight of the pseudo-label in the loss function: , in, is the adjustment parameter, is a pseudo-label sample The weight in the training loss. Samples with higher confidence and lower entropy will have higher weights, and vice versa. This uncertainty calibration mechanism ensures that during each iterative optimization process, the model is more inclined to trust those pseudo labels that are stable and have high confidence.

[0039] The loss function after uncertainty calibration is: , This can effectively improve the robustness of the self-training process and ensure that the model is not affected by incorrect pseudo-labels.

[0040] Furthermore, after several rounds of self-training iterations, the model It will be able to better adapt to unlabeled data and improve label generation capabilities based on high-quality pseudo labels. The output of the optimized model will provide a solid label foundation for the next step of multimodal fusion, making the label generation invention more stable and efficient when processing complex multimodal data.

[0041] The final output is the model parameters after uncertainty calibration and optimization. The model can not only make accurate predictions based on a small amount of labeled data, but also gradually improve the generalization ability of the model with the help of high-confidence pseudo-labels, thus laying a solid foundation for subsequent multimodal data fusion and dynamic label generation.

[0042] Step S3: Map the multimodal data in the unlabeled dataset through a shared embedding layer, and use a weighted fusion mechanism to dynamically adjust the weights of each modality according to its importance in a given task, and then perform weighted fusion to obtain a unified multimodal representation.

[0043] Specifically, in order to ensure that data of different modalities can be represented in a unified semantic space, the present invention first maps the data of each modality through a shared embedding layer: , in is the learned transformation matrix that aligns features of different modalities to the same dimension , for subsequent fusion operations. This step ensures that data from all modalities are projected into the same semantic space.

[0044] Furthermore, in order to effectively fuse information from different modalities, the present invention adopts a weighted fusion mechanism to dynamically adjust the contribution of each modality according to its importance in a given task.

[0045] For each modality, calculate its weight for the current task. , modal Weight The feature vector of each modality and the shared weight vector The inner product is determined by: , This ensures that on different samples, the weights of modalities such as text, images, and audio can be adaptively adjusted according to their relevance to the task.

[0046] Furthermore, the embeddings of all modalities are fused by weighted averaging to generate the final multimodal unified representation : , This formula dynamically fuses the features of each modality by combining the weights of different modalities, so that important modal information is enhanced and minor modal information is suppressed.

[0047] The fused representation It is a multimodal unified semantic representation that integrates data from different modalities such as text, images, and audio. This representation can capture the interactive information between multiple modalities and form a more comprehensive semantic understanding.

[0048] Output: Unified Representation The output of this step will be used in the next step of the dynamic label generation model to help generate more accurate labels.

[0049] This step ensures the effective fusion of multimodal information, and at the same time, through the adaptive weight mechanism, the contribution of each modal data can be flexibly adjusted, thereby generating a more semantically compliant multimodal representation. The whole process is concise and efficient, and is closely connected with the previous steps.

[0050] Step S4: Generate label distribution using a dynamic label generation network based on the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model.

[0051] Specifically, in the label generation process, the present invention will be based on multimodal representation A dynamic label generation network is used to generate label distribution. In order to ensure the flexibility and accuracy of label generation, the present invention needs to introduce an adaptive mechanism so that the model can adjust the label generation strategy under different inputs.

[0052] Furthermore, a multimodal label generation function is defined , whose input is a multimodal unified representation , the output is the label probability distribution for this sample . This distribution is generated by the Softmax function: , in, , is the number of label categories. The Softmax function converts the generated results into a probability distribution, indicating the possibility of each label.

[0053] Furthermore, based on the label distribution of each sample ,This paper designs a dynamic Top-K label selection mechanism to ensure that the generated labels can be adaptively selected based on the multimodal features of the current input.,For example, for samples with complex or ambiguous semantics, the model can select multiple labels; for samples with clear,semantics, only the most likely label is selected.

[0054] The dynamic label selection process selects the top label with the highest confidence. tags to implement: , in The size of can be adjusted dynamically, depending on the complexity of the current sample and the confidence of the model in the sample. This mechanism allows the label generation process to flexibly generate labels that adapt to the scene according to the changes in the input multimodal information.

[0055] Furthermore, in order to ensure the adaptability of label generation, the present invention introduces two important innovative mechanisms: Modality uncertainty weight mechanism: In the label generation process of multimodal data, information of different modalities may have different confidence and uncertainty. The present invention calculates the prediction uncertainty of each modality and regards uncertainty as an important component of the label weight in the label generation process.

[0056] Uncertainty calculation: using the output probability distribution of each mode To estimate the uncertainty. For each mode Representation , whose uncertainty is calculated by entropy: , in Representing modality For Category The predicted probability of . A high entropy value means greater uncertainty.

[0057] The present invention introduces uncertainty into the label generation process, ensuring that the modality with high uncertainty contributes less to the final generated label, while the modality with high confidence plays a greater role. By adjusting the weight of each modality, the present invention defines the adaptive weight of multi-modality: , in is a balancing parameter used to control uncertainty. When the entropy of a mode is low (i.e., the confidence is high), its weight When it is close to 1, the model will trust the prediction of this mode more; otherwise, the influence of this mode will be reduced.

[0058] Furthermore, in order to ensure the consistency of labels generated between multiple modalities, the present invention introduces a modality consistency loss to constrain the differences between label distributions generated by different modalities. Specifically, for the label distribution generated by each modality , the present invention hopes that they can be consistent with each other. Definition of consistency loss: , This loss term ensures that they generate consistent label distributions by minimizing the prediction differences between different modalities, thereby improving the semantic relevance across modalities.

[0059] Ultimately, the optimization process of label generation needs to design a loss function that combines the prediction error and modal uncertainty of the label distribution and the adaptive mechanism. The complete loss function consists of cross entropy loss, modal consistency loss, and regularization term: , in, is the cross entropy loss of the label, which measures the difference between the generated label and the true label; It is the modality consistency loss, which ensures that the labels generated by different modalities are consistent; is a hyperparameter that controls the weight of the loss term and is used to adjust the impact of different mechanisms on model optimization. This loss function balances the error of label generation, modal consistency, and modal uncertainty to ensure that the generated labels can accurately capture multimodal information and adapt to the characteristics of different modal data.

[0060] Furthermore, after the dynamic label generation process, the output label distribution Represents the label probability distribution of each sample. is the result of combining multimodal information. This label distribution will be used as the input of step 5: interpretable label generation and feedback optimization.

[0061] Step S5: Make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model adjusted by user feedback.

[0062] Specifically, in order to make the labels generated by the model more transparent and interpretable, the present invention designs an explanation generation mechanism based on attention weight and feature contribution analysis. ,The present invention needs to explain the generation process of each label and show the relationship between the label and the input data (including multimodal information such as text, images, audio, etc.).

[0063] Multimodal attention weight explanation: In step 4, the model generates labels by fusing different modal information through dynamic weight adjustment. The present invention can use these weights to explain the contribution of each modality in label generation. Specifically, for sample , which generates the label Multimodal attention weights Having been calculated in step S4, these weights can be directly used to explain the process of label generation: , These weights reflect the relative importance of text, images, and audio in generating the final label. By displaying these weights, users can intuitively see the contribution ratio of different modalities to label generation.

[0064] Furthermore, in addition to the contribution analysis at the modality level, the present invention also needs to show the specific role of each feature in label generation. The present invention uses the Shapley value method to analyze the contribution of multimodal features in each label generation process. , its multimodal features The contribution of each dimension feature in is calculated by Shapley value, and the formula is: , in represents the index of the feature, Features For samples Through Shapley value analysis, the present invention can show which features play a key role in the generation of each tag, helping users understand the generation logic.

[0065] Furthermore, after providing interpretability of label generation, users can provide feedback on the generated labels. The user feedback mechanism allows users to adjust the labels of each sample and provide reasonable explanations to help improve the model.

[0066] Implementation of feedback mechanism: Users can evaluate the generated labels through an interface, and labels can be confirmed, modified or rejected. The specific feedback data is recorded as ,in Label for user feedback.

[0067] Optimization strategy for user feedback: In order to enable the model to quickly learn user feedback and improve the label generation process, the present invention designs a feedback optimization strategy based on reinforcement learning. After receiving user feedback, the present invention updates the parameters of the model by comparing the feedback information with the label generated by the model, using the following update formula: , in is the learning rate, is the loss function based on user feedback, defined as: , The loss function adjusts the model parameters by measuring the difference between the labels fed back by users and the labels generated by the model. , so that the next generated label can better meet the user's expectations.

[0068] Furthermore, after each user feedback, the model will update parameters according to the feedback, so that the model can be gradually improved. Through multiple rounds of feedback-optimization cycles, the label generation ability of the model will be continuously improved. At the same time, the present invention can continuously optimize its interpretation generation mechanism according to feedback, providing a more transparent label generation process.

[0069] Furthermore, the present invention generates labels and provides interpretable information (such as modal contribution and feature contribution).

[0070] User feedback: Users confirm or adjust the generated labels.

[0071] Model optimization: Based on user feedback, the model updates parameters .

[0072] Next round of generation: In the next round, the model generates labels that are more in line with user expectations based on the updated parameters.

[0073] Furthermore, after multiple rounds of iterative optimization, the final output is a label generation model adjusted based on user feedback. The model not only generates high-quality, multimodal-aware labels, but also has good interpretability and user interactivity, which can meet the needs of practical applications. The output result is the model It will become the final model of the present invention, with the ability to generate accurate labels and dynamically adapt to user feedback.

[0074] Another embodiment of the present invention provides an intelligent text data tag generation system, the system comprising: A few-sample data training module is used to obtain a text dataset extracted from a small number of annotated samples, establish a label candidate model based on a few-sample and sparse data loss function according to the obtained text dataset, and introduce a multi-level label generation mechanism in the label candidate model. The label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model; The label candidate model optimization module is used to input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution, and at the same time, design a pseudo-label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo-labels for the screened samples, and then combine the generated pseudo-labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model; The pseudo-label selection strategy designed for unlabeled data is used to screen unlabeled samples, and the specific steps include: Design a confidence threshold and compare it with each unlabeled sample. When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label. Otherwise, the sample is skipped to avoid introducing erroneous labels with low confidence. If the highest probability predicted by the label candidate model is less than the confidence threshold, the sample will not generate a pseudo label. The data fusion module is used to map the multimodal data in the unlabeled dataset through a shared embedding layer. At the same time, a weighted fusion mechanism is adopted to dynamically adjust the weights of each modality according to its importance in a given task and then perform weighted fusion to obtain a unified multimodal representation after fusion. The label distribution module is used to generate label distribution using a dynamic label generation network based on the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model; The adaptive mechanism of label generation includes a modality uncertainty weight mechanism and an inter-modality consistency constraint; the modality uncertainty weight mechanism calculates the prediction uncertainty of each modality and takes uncertainty as an important component of the label weight in the label generation process; the inter-modality consistency constraint is used to constrain the differences between label distributions generated by different modalities; The label optimization generation module is used to make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model after adjustment based on user feedback.

[0075] In summary, the present invention effectively overcomes the shortcomings of existing label generation technology in data scarcity, multimodal processing and transparency by combining three major innovative points: few-sample self-training, multimodal fusion and explainable AI, and provides an intelligent, efficient and reliable text data label generation method and the present invention, which has broad application prospects.

[0076] These are just some preferred embodiments of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for generating intelligent text data labels, characterized in that: The method comprises the following steps: Step S1, obtaining a text data set extracted from a small amount of annotated samples, establishing a label candidate model based on a few samples and a sparse data loss function according to the obtained text data set, and introducing a multi-level label generation mechanism in the label candidate model, wherein the label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model; Step S2: input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution, and design a pseudo label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo labels for the screened samples, and then combine the generated pseudo labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model; The pseudo-label selection strategy designed for unlabeled data is used to screen unlabeled samples, and the specific steps include: Design a confidence threshold and compare it with each unlabeled sample. When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label. Otherwise, the sample is skipped to avoid introducing erroneous labels with low confidence. If the highest probability predicted by the label candidate model is less than the confidence threshold, the sample will not generate a pseudo label. Step S3: Map the multimodal data in the unlabeled dataset through a shared embedding layer, and use a weighted fusion mechanism to dynamically adjust the weights of each modality according to its importance in a given task, and then perform weighted fusion to obtain a unified multimodal representation. Step S4: Generate label distribution using a dynamic label generation network according to the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model; The adaptive mechanism of label generation includes a modality uncertainty weight mechanism and an inter-modality consistency constraint; the modality uncertainty weight mechanism calculates the prediction uncertainty of each modality and takes uncertainty as an important component of the label weight in the label generation process; the inter-modality consistency constraint is used to constrain the differences between label distributions generated by different modalities; Step S5: Make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model adjusted by user feedback.

2. The method for generating intelligent text data tags according to claim 1, characterized in that: In step S1, in order to process the extracted text dataset, it is necessary to map the text into a high-dimensional vector space through a pre-trained language model; The loss function based on sparse data is designed as follows: First, define a label candidate model , used to embed Generate preliminary label candidates, as follows: ， in, is the initial label candidate generated, represents the number of potential labels; Design a regularized loss function to deal with the few-sample problem, as follows: ， in, is the cross entropy loss, which measures the difference between the true label and the predicted label. is the true label, N is the number of labeled samples, i is the current sample, is the regularization coefficient, is the regularization term; The multi-level label generation mechanism is shown as follows: Each text embedding By labeling candidate models Generate preliminary label probability distribution , by introducing a multi-class label activation mechanism to generate a preliminary label candidate set to enhance label diversity, as shown below: ， in, Each element in represents the probability of generating a certain label. is the Softmax function; At the same time, in order to deal with semantic ambiguity and uncertainty, the selection mechanism is defined as: ， That is, choose the first The label with the highest probability is taken as the preliminary label candidate. is the first K selected tags, indicating the tag set sorted by confidence. In the confidence sorting, select the indexes of the K tags with the highest confidence.

3. The method for generating intelligent text data tags according to claim 2, characterized in that: The regularization term According to the adaptive adjustment of the confidence of each feature, the largest part of the discrete features is penalized to avoid overfitting of small sample data in high-dimensional space, which is expressed as follows: ， in, Indicates The model weights of feature dimensions; is embedded The Dimensional features; and They are the embedded features in The mean and standard deviation of the dimension, d is the number of features, j is the index of the feature, is the exponential operator symbol; Among them, when Closest When , the regularization effect is the strongest, indicating that this feature dimension is the most uncertain.

4. The method for generating intelligent text data tags according to claim 1, characterized in that: The step S2 further includes: A self-training loss function is designed according to a new training set to train the label candidate model. The new data set includes annotated data and pseudo-label data with high confidence. At the same time, an additional penalty term is added to the pseudo-label data to reduce the noise that may be caused by the pseudo-label. The self-training loss function is expressed as follows: ， in, To mark the number of samples, is the number of pseudo labels generated; is the cross entropy loss, which measures the difference between the true label and the predicted label; It is a newly introduced pseudo-label uncertainty penalty term, which aims to weight the pseudo-label according to its confidence level to prevent pseudo-labels with low confidence from adversely affecting model training; and are important coefficients used to balance the labeled data and pseudo-label data; the pseudo-label uncertainty penalty term It is expressed as follows: ， in, Represents unlabeled samples For The predicted probability of each category, C is the total number of categories; In the process of label candidate model, an uncertainty calibration is performed after each iteration. The uncertainty calibration adjusts the weight of the pseudo label for model training according to the prediction confidence of the pseudo label and the stability of the model to the label. And the entropy value of the label Together they determine the weight of the pseudo-label in the loss function, expressed as follows: ， in, is the adjustment parameter, is a pseudo-label sample The weight in the training loss; samples with higher confidence and lower entropy will have higher weights, and vice versa; Then the optimized self-training loss function is It is expressed as follows: 。 5. The method for generating intelligent text data tags according to claim 1, characterized in that: The weighted fusion mechanism specifically comprises the following steps: For each modality, calculate its weight for the current task, for the sample , modal Weight The feature vector of each modality and the shared weight vector The inner product is determined as follows: ， in, is the mapping of the modality m in the embedding layer; the weighted fusion mechanism ensures that the text ,image , Audio Ability to adaptively adjust weights based on their relevance to the task.

6. The method for generating intelligent text data tags according to claim 1, characterized in that: The step of using a dynamic label generation network to generate label distribution according to the fused unified semantic representation specifically includes: Define a multimodal label generation function , whose input is a multimodal unified representation , the output is the label probability distribution for this sample , the distribution is generated by the Softmax function and is expressed as follows: ， in, , the Softmax function converts the generated results into a probability distribution, indicating the possibility of each label; Based on the label distribution of each sample A dynamic Top-K tag selection mechanism is designed to ensure that the generated tags can be adaptively selected according to the multimodal features of the current input. The dynamic tag selection process of the dynamic Top-K tag selection mechanism selects the top-K tags with the highest confidence. This is achieved by using a tag, as shown below: ， in The size can be adjusted dynamically. is the first K selected tags, indicating the tag set sorted by confidence. In the confidence sorting, select the indexes of the K tags with the highest confidence.

7. The method for generating intelligent text data tags according to claim 4, characterized in that: In step S4, uncertainty is introduced into the label generation process to ensure that the modality with high uncertainty contributes less to the final generated label, while the modality with high confidence plays a greater role. The uncertainty is based on uncertainty calculation, which uses the output probability distribution of each modality. To estimate the uncertainty, for each mode Representation , whose uncertainty is calculated by entropy and is expressed as follows: ， in, Representing modality For Category The predicted probability of , C represents the total number of categories, is the uncertainty measure of label j of modality m; Design multi-modal adaptive weights and dynamically adjust the trust of the label candidate model on each modality by adjusting the weight of the modality, as shown below: ， in, is the adaptive weight of modality m on label j, It is a balance parameter used to control uncertainty. When the confidence of a certain mode is the highest, its weight If it is closest to 1, the label candidate model will trust the prediction of this modality the most; otherwise, the influence of this modality will be reduced; The inter-modality consistency constraint is calculated based on the modality consistency loss, which is expressed as follows: ， in, represents the consistency loss, Representing modality The output probability distribution of Representing modality The output probability distribution of .

8. The method for generating intelligent text data tags according to claim 7, characterized in that: Finally, the adaptive mechanism of label generation is combined with the prediction error to further optimize the loss function of the optimized label candidate model. , which is expressed as follows: ， in, is the cross entropy loss of the label, which measures the difference between the generated label and the true label; It is the modality consistency loss, which ensures that the labels generated by different modalities are consistent; It is a hyperparameter that controls the weight of the loss term and is used to adjust the impact of different mechanisms on model optimization. M is the total number of samples.

9. The method for generating intelligent text data tags according to claim 1, characterized in that: The steps of making the labels generated by the label candidate model transparent and explainable include: For each sample , explain the generation process of each label, and show the relationship between the label and the input data according to the mode in the input data and the corresponding weight; At the same time, the Shapley value method is used to analyze the contribution of multimodal features in each label generation process, and then the specific role of each feature in label generation is demonstrated, as shown below: For samples , its multimodal features The contribution of each dimension feature in is calculated by Shapley value, and the formula is: ， in, represents the index of the feature, Features For samples The contribution of tag generation.

10. An intelligent text data label generation system, characterized in that: The system comprises: A few-sample data training module is used to obtain a text dataset extracted from a small number of annotated samples, establish a label candidate model based on a few-sample and sparse data loss function according to the obtained text dataset, and introduce a multi-level label generation mechanism in the label candidate model. The label generation mechanism embeds labels and semantics, and then outputs a preliminary trained label candidate set through the label candidate model; The label candidate model optimization module is used to input the unlabeled data set into the label candidate model to generate a preliminary label probability distribution, and at the same time, design a pseudo-label selection strategy for the unlabeled data to screen the unlabeled samples, generate pseudo-labels for the screened samples, and then combine the generated pseudo-labels with the original labeled samples to form a new training set to iteratively optimize the label candidate model to obtain the optimized label candidate model; The pseudo-label selection strategy designed for unlabeled data is used to screen unlabeled samples, and the specific steps include: Design a confidence threshold and compare it with each unlabeled sample. When the probability value of a label predicted by the model is greater than the threshold, the label is retained as a pseudo label. Otherwise, the sample is skipped to avoid introducing erroneous labels with low confidence. If the highest probability predicted by the label candidate model is less than the confidence threshold, the sample will not generate a pseudo label. The data fusion module is used to map the multimodal data in the unlabeled dataset through a shared embedding layer. At the same time, a weighted fusion mechanism is adopted to dynamically adjust the weights of each modality according to its importance in a given task and then perform weighted fusion to obtain a unified multimodal representation after fusion. The label distribution module is used to generate label distribution using a dynamic label generation network based on the fused multimodal unified representation, design an adaptive mechanism for label generation to ensure the adaptability of label generation, and finally combine the adaptive mechanism for label generation with the prediction error to further optimize the loss function of the optimized label candidate model; The adaptive mechanism of label generation includes a modality uncertainty weight mechanism and an inter-modality consistency constraint; the modality uncertainty weight mechanism calculates the prediction uncertainty of each modality and takes uncertainty as an important component of the label weight in the label generation process; the inter-modality consistency constraint is used to constrain the differences between label distributions generated by different modalities; The label optimization generation module is used to make the labels generated by the label candidate model transparent and explainable, then collect user feedback on the generated labels, optimize the label candidate model and explainability based on the user feedback collected each time, and finally output the labels generated by the label generation model after adjustment based on user feedback.

Images